Slashdot Mirror
AI to Monitor Foreign Press for Threats
jefu writes "According to the New York Times, the US Department of Homeland Security is funding AI tools to monitor the foreign press in order to detect threats to the United States. While the article says there are restrictions on doing this kind of monitoring within the US, there are no restrictions on media outside the US. (No hint is given as to how this would apply to syndicated articles written in the US and published abroad.) This is as yet experimental."
What's wrong with using what is traditionally referred to as Open Source Intelligence (OSINT) against publicly available sources?
This has been done for years, and is a time-honored and respected mechanism for gathering intelligence. What's wrong with then leveraging technology to more effectively search larger volumes of information and weed out individual pieces of information for further analysis, to identify trends, and so on?
The Open Source Center, formerly the Foreign Broadcast Information Service, already does this with foreign broadcast media, and is able to collect and transcribe, on the fly, information from foreign radio and television broadcasts in a variety of languages and dialects with incredible accuracy, and then make the resultant material searchable. The new initiative would go one step further and apply artificial intelligence techniques to automated searching, that can more easily target and bring to light trends or time-critical information.
Different business and governmental entities do this globally; it's traditionally referred to as "current awareness", and many academic and corporate entities offer current awareness services. All of these services will leverage technology, live realtime searching and alerting, and so on, to make the information more timely, valuable, and relevant.
Remember, this is publicly available and published information.
Also, submitter is a little misguided when he says "No hint is given as to how this would apply to syndicated articles written in the US and published abroad." That misunderstands the purpose of this; the program is designed to look at foreign media sources as one component of OSINT, because they are a a valuable source of such information, and can reflect local trends and patterns, and may reveal changing or growing (or waning) sentiments on particular topics on the part of a local populace or media outlet, or even a government in the case of state-controlled media. We generally don't get that kind of information from US-based media, and this has nothing to do with whether US-based media outlets publish abroad. It's already public information and has been published publicly. The restrictions are geared to prevent an appearance of overt US press monitoring.
OSINT is a one-way source of intelligence information: from it, to the gathering entity. Any assumptions that the viewing of already-public information then implies that there will be a commensurate attempt to silence such information (especially when the information isn't under our control, and ignores the fact that we can't "silence" things like Iran's state media) both makes a a fallacious logical leap and grossly misunderstands the purpose and scope of OSINT.
All the critics can say is that it's "creepy and Orwellian," but of course, there's nothing wrong with the government or its intelligence components reading, viewing, or collecting publicly available and indeed overtly publicly published information. The intelligence community gets ripped when it doesn't gather enough information, and will no doubt get ripped for gathering "too much" in a "creepy" way, even when it's from overtly and intentionally public sources, and especially if it uses technology to do it.
There is a real concern about the growing use of automated and electronic intelligence gathering in lieu of human intelligence, but ultimately, both are valuable. Unfortunately, electronic and signals intelligence is often much more costly, and sometimes gets more attention in some parts of the intelligence community while human intelligence needs languish.
And so, SkyNet became self-aware and bombed al Jazeera and the New York Times.
According to the New York Times, the US Department of Homeland Security is funding AI tools to monitor the foreign press in order to detect threats to the United States.
"We keep it, Sir, but it still comes up with the number one threat to the US is Donald Rumsfeld."
A feeling of having made the same mistake before: Deja Foobar
What issue would that be exactly? If media is releasing information, how can there be an issue with the government reading that information, parsing it rhought AI or lining bird cages with it for that matter? I could imagine there might be an issue with putting out false information to domestic press, ie PsyOps, but monitoring public source information seems very much a no-brainer.
"Don't you know you're going to shock the monkey?"- Peter Gabriel
Remember, when grep is outlawed, only outlaws will have grep
The Independent: Reverend Spooner Arrested in Friar Tuck Incident - ISIHAC, Historical Headlines
Eh... is anyone REALLY woried about the US government reading press from around the world? I mean... once you sell something on the street or post it to the web you kind of assume that everyone can read it.
I have a feeling that what this program really is doing is looking for "major events" as fast as possible. So, if a news agency reports in London that an airplane just blew up, an AI in the US shouts out a warning to its operator. This way, events that might signal something for the US worry about are brought to light within minutes instead of hours. In the case of an airplane blowing up in the UK, it might signal that a larger operation was on to blow up American airplanes as well. This way, you can start assessing the threat right away and decide if anything should be done.
Such a program could also act as a political heads up. If a Pakistani papers is reporting that a coo is in progress, that is a damn nice thing to know ASAP so that you can decide how to deal with a nuclear armed nation with a collapsing government.
I am sure that the US has piles of people already scanning newspapers from around the world, I imagine that this AI is simply an attempt to cut delays down from hours to minutes.
No...all those foreign papers have them...you know the personals for terrorists...
SMET seeking like minded individuals for cave expeditions, vigorous exercise, and praying to Allah. Enjoys long nights plotting together and holding hostages.
For those who missed it, Single Middle Eastern Terrorist (SMET).
"Some days you just can't get rid of a bomb."
There are several pro-Israel monitoring services watching the press - CAMERA in the US, BICOM in the UK, and MEMRI to monitor the Arab press. CAMERA is noted for having a good database of stories about Israel. Apparently stories mentioning Israel are found automatically, but evaluated by people.
The CIA has something called the "Open Source Intelligence Service", which started as the "Foreign Broadcast Monitoring Service". Visualize some poor guy spending eight hours a day listening to Radio Albania, all through the Cold War. Most of the content is terminally boring, but then, one day the announcer says "so we're invading Yugoslavia", and the CIA needs to notice this. There have been repeated attempts to automate the job.
No but the domestic press is useless in this case. They have become a de facto public relations firm for the government.
Yeah, but it's not their own government.
Why would the USA government bother to read the press when it is already writing it ? :P
I have discovered a truly marvelous proof of killer sig, which this margin is too narrow to contain.
Server 986762 has red flaggged the following sir. It's a little old but I think it may be highly relevent: Top Secret Report
Because you can - or because you should?
Isn't that exactly what the CIA was set up to do?
I wonder if they are doing anything more than searching Google News each morning.
The first thing I thought on reading this headline was "Who the hell is Al? You mean the sidekick from Home Improvement?"
I guess he would occupy the flannel office.
Find environmentally and socially responsible products on http://buy-right.net
It's better than just major events. You can apply stats to huge volumes of data, find all sorts of interesting correlations. Such as personals adverts to signal events, stock market rises/falls prior to terrorist events.
Deleted
Thanks again to the corporate-political machine for letting us know what utter and complete contempt you hold us in. And no, this isn't just a dig at Bush or his administration. It's direceted at each and every politician who's been bought and paid for by the corporate and special interest lobbies - in other words 95% of them.
It is way, way past time for this to stop. And it is up to us to stop it.
What if the Hokey Pokey really is what it's all about?
That we have AIs that can reliably understand Arabic, Urdu, etc., because we sure don't have enough real people that can.
Some mornings it's hardly worth chewing through the restraints to get out of bed.
I think it's a little more subtle than that.
There is a theory, which I have heard articulated from time to time (although I don't know if there's a name for it or not) which says that right before a major event there is a lot of "chatter"; subtle yet distinct signs that something is about to happen, but which are too minor on their own to generate any attention. Once the event happens, in hindsight you can look back and recognize them. It's sort of a reverse butterfly-effect; the assumption is that no matter how good at being secretive you are, you will make some signs in the course of executing your plot, and some of those signs will percolate and be reported in papers somewhere. So you just need to know what to look for.
So basically what you might do, is take a big pattern-matching AI system, and "teach" it using the media records preceding other big terrorist events. 9/11, London, Madrid, etc. You have it comb through all the world media before those events, and see if you can find patterns, the little things that in retrospect might have alerted you that something was up. Then, thus primed with information and hopefully some patterns, you set it loose on the real-time news feeds.
In theory -- if the theory holds water, anyway -- the system might then be capable of giving you a warning of something big heading your way, picking up on stuff that a person might not recognize.
Anyway, I'm not sure if that's the theory that this particular system is going to try and use, but it's one idea that I've heard described; sort of as the 'holy grail' of machine-derived OSINT. More likely, you'd end up with a system that just gives you statistical summaries of the number of anti-US editorials in various countries or something. Useful for the State Department perhaps, but I'm not sure for preventing the next 9/11.
"Ladies and gentlemen, my killbot features Lotus Notes and a machine gun. It is the finest available."
I think that many people are hostile towards this kind of system not because of what it is (basically a computer indexed newsmedia database), but who has control over it - and more importantly, who doesn't.
Imagine the web without search engines, just a bunch of sites connected by links. Then, imagine that the CIA created a massively expensive and elaborate project (call it project GOOGLE) to index, cache, and analyze all the content on the web, and make it searchable. Of course, project GOOGLE would be highly guarded and only government officials could use it, leaving everyone else to wonder uneasily what the government was learning about them through their database.
That would be creepy, because it would mean the government would have an effectively God-like view of all activity on the web, while everyone else would be in the dark. Make such a project publically available and transparent (like the real Google), and it's accepted almost universally. Of course, you could argue that people should build their own AI-driven newsmedia uber-database, but the reality is that the US government is probably one of the few entities with the massive resources and will needed to create something like this, which means that by definition the rest of us will be left in the dark.
In a nutshell, I think the crux of the issue is that folks don't trust the US government, so every time the government proposes a new way to expand their power, people immediately jump on the ways that it could be abused. This, I believe, is as it should be.
Why would you ask an AI to look for threats if you are not even willing to listen to your close friends and allies?
q .fries/
Read for instance this letter from the French Ambassador to the United States in 2003
http://www.counterpunch.org/levitte02142003.html
It clearly warns about the mess a war in Irak would get us all in to.
It states that Iraq is not a threat and it predicts the rise in terrorism worldwide, the destabilisation of the region and the civil war in Irak that we see now.
One month after this the whole "freedom fries" thing started.
http://www.cnn.com/2003/ALLPOLITICS/03/11/sprj.ir
This AI will just add data to the heap that is allready ignored.
I concur with your concerns over this technology. It seems to me that this could very easily be a beta-test for software that the government will then begin to use to search through the mass of information that that the NSA collects every day on millions of Americans. One thing that I haven't seen mentioned in this discussion is an old intelligence adage that goes basically like this, "You need to base your threat analyses on capabilities not intentions." So, from a threat to civil liberties viewpoint, this program represents a potential huge step forward in the ability of the U.S. Government to monitor the communications of its citizens and automatically flag anything that "looks like a threat". When one then adds the law that Congress just passed concerning who may be declared an "illegal enemy combatant" (which now includes U.S. citizens) our government now has the capability to monitor our email or other electronic communication, analyse it with software which no one will know anything about regarding false-positive rates, failure modes, etc., then use that analysis to declare a citizen to be a "threat" and thus an "illegal enemy combatant", then arrest that person, strip them of their civil liberties, and turn them over to the gentle care of military tribunals which may, at the discretion of the tribunal judge, allow the use of "coerced self-incriminatory testimony". Maybe I'm just paranoid, but this sounds an awful lot like Orwell's description of the the functions of the "Thought Police" in "1984".
Just my $.02,
Ron
Impeach Barack Obama for violating the Constitutional requirement to be a "natural born" citizen to hold the office of P