Slashdot Mirror
More Details of the NSA's Social Network Analysis
mrogers writes "USA Today has a story describing how the NSA looks for suspicious calling patterns in the huge volumes of traffic data it collects. "Templates" such as a call from overseas followed by a flurry of domestic calls are used to identify leads, which are forwarded to the FBI for investigation. There have been complaints that low-quality leads are drawing agents away from other cases, and similar pattern-matching approaches have been found wanting in the past. Can data mining identify terrorists?"
And don't tell me That's just television because no, sir, It's not TV, it's HBO.
For more info, see here...
Here's what I do: Bitty Browser & Andromeda
They should formalize this practice and make a palindrome out of the resulting acronym. That way we can be distracted with how cool they are to think of such things instead of worrying about what they're actually doing.
NSA-ASN - NSA's Analysis of Social Networks.
*sigh* I'm very honestly starting to get a sick feeling in my stomach over the direction our (my) country is headed. And yet, I feel like there's nothing I can do about it. Vote? Yeah... right.
A community-oriented lyrics site
From now on, I'm using world of warcraft to plan my activities.
The problem is, this strategy is not only ineffective, it can be counterproductive.
There is plenty out there on the "Strength of weak links", where past associations (old roommates, sleeper cells), with not contact can be very strong service links when reinitiated.
There is also plenty out there on how this is DoSing the FBI.
And the tin foil hat crowd (a very popular piece of headware these days) will point out that this tool is far more useful for targeting individuals than searching for patterns. And what if you are the target?
Test your net with Netalyzr
If this wholesale data mining works, then the government will tout this success as justification for its acts. If it doesn't work, the government will complain that we're not letting them do enough to ensure our safety, and use the failure to justify even more outrageous violations of our privacy.
Whether it works or not, however, is beside the point. The point is: is it legal? Enough people have maintained that it is not to warrant a serious investigation into the matter.
____
~ |rip/\/\aster /\/\onkey
The monitered person can distribute the calls through multiple phone lines. With cooperation, a group of individuals can pool phones to use and this system won't detect them. What is detectible is how many phone lines are registered to a person.
However the government has yet to catch up to the real world. I can disitalyl distribute the message through the internet using techniques that would not arouse suspicion, partivularly with al the online gaming of today.
Roger wilco anyone?
Slashdot's rate-of-post filter: Preventing you from posting too many great ideas at once.
Aside from this being patently illegal, what bothers me is the cavalier attitude behind it, and the fact that it is already being abused to track down people who aren't terrorists, but who are merely doing their job to keep government entities like the NSA under some semblance of control - the journalists. There is no end to the manner in which this kind of information could be abused.
She's always getting calls from various places and then making a flurry of more local calls. She uses code phrases like "your cousin's baby was born last night and it's a boy", or "Great Aunt Zelda had a stroke but they say she's going to be okay".
"Prefiero morir de pie que vivir siempre arrodillado!"
"Hey Akbar, just calling to let you know Mohamed and Alimah just had a healthy baby boy!"
"Oh great, I'll let the family over here know!"
*meanwhile, in the basement of a bunker somewhere*
"My God! It's nine eleven times ten thousand! Nine million one hundred and ten thousand!"
Whoever said this was about "terrorists"?
A country of 300 million people cannot have that many actual terrorists in it, even if you count domestic lunies like Timothy McVeigh and the Unabomber in the category (or more accurately the next generation of bomb making lunies). Monitoring a sizable fraction of that 300m can't possibly be just about finding "terrorists" - for one thing it's a needle in a haystack, and for another the number of other uses/abuses of such a system are too many to count.
Bet good money that most of the people who are or will be advesely affected by this surveilance have little or no connection with terrorism. Even if there was once some noble intent of protecting people by finding monsters hidden among them, it won't just be used for that. Any time you have a major source of power in polical hands, you can bet on it being abused eventually - and what greater power over a domestic population is there than widespread spying without judicial oversight?
Erotic is when you use a feather. Exotic is when you use the whole chicken.
"Can data mining identify terrorists?"
No. It can identify people who have calling patterns associated with terrorist activity, regardless of whether they are a terrorist or not.
Note that these calling patterns cannot be used to associate that person with a committed or planned crime in the normal data mining scenario.
Data mining is unreasonable search.
Now, I have no problem if they've got evidence of a crime or plan of a crime, and use known information to deduce who might else be involved. That's investigative work.
Data mining is speculative work, not investigative, so regardless of whether it *can* be used for speculative 'research' into the activity of American citizens, it *shouldn't* be.
"Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
This approach to finding patterns works well in marketing where getting a 1% rate of sales to contacts is a good response rate. The problem with using this approach for anything in the real world is the 99% of the time you're wrong.
They looked at the history of a few people and found a pattern. Now that the pattern has been disclosed, only historical information is likely to have any merit. If the people controlling the communications know this is a way to be found, after getting a call from a watched country, they'll have the people go somewhere else and send emails or otherwise use a different channel for communication.
Knowing all of the data points isn't enough if you don't know which ones in different databases (phone, email, etc) are related and why.
This won't work at all.
They are operating under a logical fallacy. A flurry of calls after an overseas call does not mean the two are related in any way. Perhaps (and more likely than the person being a terrorist) is that the person which received the overseas call and then calls domestically is just relaying family information.
I know my family operated like this (although completely within the US). All you had to do was tell my grandmother something, and you could rest assured she'd spread the news to the rest of the family for you.
What ever happened to "Live free or die", "Give me liberty or give me death", or "Those who are willing to sacrifice their basic liberties to assure their security deserve neither."?
Those quotes are not just platitudes... they are *good ideas*.
Keep the canned patriotism, give me my rights, and I'll just take my chances.
Another consultant who stuck it out.
"We are the Priests, of the Temples of Syrinx..."
The last question in the post is ill posed: can data mining find terrorists -- the answer is yes. Simply set the threshold low: select anyone who has used a phone at any time and you'll likely get most terrorists. The problem is not sensitivity -- the real problem is specificity. If you have no or low specificity then the FBI will be investigating everyone (even those who "have nothing to fear since they have nothing to hide"). Specificity is where the search process interfaces with the Bill of Rights on right to privacy and protection from unlawful search and seizure. High specificity would allow the courts to work by granting warrants; low specificity degenerates into witch hunting.
It is easy to spot "distinct patterns" after you know all the players and can put the pieces together in context. As they say, Hindsight is 20/20.
I have a sister over-seas. If/when she calls anyone else in the family with news/updates/etc it will generate this pattern of many domestic calls as we have a large extended family who wants to know how she and her family is doing.
This does not mean we are terrorist, even though we might fit this "pattern" of suspicious calls. I bet calls to 900 numbers are suspicious and need lots of monitoring as well.
Many ways to abuse this.
I only look human.
My mother is a halfling and my dad is an ogre, so that makes me an Ogreling
Dismissing the legality and morality of doing this...
Let's look how most Network Intrusion Detection Systems work today, including the OSS favorite Snort.
We start off with a bunch if signatures. These signatures are analyzed against including network traffic. A signature is matched, an alert is sent out (syslog, mysql, whatever) and my little console displays the alert. I analyze, determine it's a "false alert". I try to tune it out, maybe, depending on frequency and annoyance, and continue on to the next (false?) alert. If the alert is deemed true, I determine if we were hacked or if something more serious is going on. Usually, I get other people involved.
Sounds like the NSA's system is very similar to the job of our favorite IDS operator. In fact, it's exactly the same thing. Some softwatre looks for patterns in telephone network traffic. Once these patterns are found, they do a quick check (basic analysis) to confirm the pattern has matched. Then, the alert is passed on to a different team to investigate whether there is a more serious event or not.
Are there false positives? Yes. Are there false negatives? Yes. Does this mean the method is ineffective? No. Does this mean it should be shut down? No. If it did, why am I, and thousands of others, getting paid for everyday?
I mean its obvious.
A band leader gets a call from a booker in Europe who wants them come play.
The band leader calls all the band members to line them up for the tour.
They cancel any local gigs that overlapped.
Those venues or bands call other bands or subs to fill those spots.
Result: The NSA gets to be first in line for tickets.
Squirrel!
How could those calling patterns ever cause false leads? Surely terrorists operate like clocks and do everything by the numbers.
Okay, here's an example of how stupid the example given is (and it's not the example that's stupid, it's the intelligence community): I'm an American I have good friends, or maybe family living overseas. Let's say my brother lives in Germany and he just called me to tell me that his wife had a baby boy. So, what am I going to do? Call everyone in my family and anyone that knows my brother well and say, "Guess what, they had a baby boy."
The fact is that, with calls between friends and family overseas in particular, the calls are not infrequently going to be some sort of major or semi-major news that the person in the States is then going to want to share with other friends and family. If the FBI is getting hit with all this garbage, I'm surprised they find time to do anything else.
I'm not saying this stuff can't be used to find terrorists, but at what expense? I would imagine there are much more effective ways to spend the money.
To bring the example a little closer to home, back in the early 90s when export restrictions on encryption were quite a bit tighter than they are now, I was asked by an uncle of mine (who's a venture capitalist) to do a little research into encryption. He had been approached by a group that had come up with some new encryption algorithm and he wanted me to get some sort of feel for how theirs stacked up.
So, I go onto Usenet and start asking some questions, trying to educate myself on this stuff. A few weeks later, I'm talking to one of my neighbors and she says, "So, did you get that job at the White House?" I said, "What job at the White House?" She said, "Well, there were some agents from the State Department here asking questions about you and they said it was for a job at the White House."
Now, I'm no rocket scientist, but I can do the math. Ask about encryption, agents show up. I suspect the two were related. I'm sure they were probably NSA agents since encryption is really more of their deal, or maybe State Dept. agents tasked by the NSA. But whatever.
Had they even looked at my file, which I'm sure they had since I had a full background check for a security clearance a few years prior, they would have quickly discovered that I'm someone of little consequence and not a likely spy. But no, they had to send out a couple agents to investigate me asking questions that anyone from anywhere around the world could have posted on Usenet. What a complete waste of time and money. And it's not like you couldn't just download regulated encryption algorithms off the net at the time anyway.
But I digress. Spending money to protect us is fine, if it's spent wisely. This is costing time of valuable people and untold amounts fo money for what is sure to be barely usable information. But hey, that should come as no shock to anyone.
So how many times has this happened. One call to aunt Martha, who then spreads the workd and then gets a visit from the FBI or agents of HS.
Undetectable Steganography? Yep, there's an app fo
There was a local news story about a terrorism suspect who was picked up locally because of a tip from a flight school. Not from monitoring his phone calls, not by fingerprinting him when he came into the country, not by spy plane, satellite or any other whiz bang technology. Just a clerk at a airport counter in the middle of bf nowhere. And that's the sensor net that offers the best hope we have of combating terrorism. The clerk at the store, the landlord they rent from, the agent at the ticket counter, the hotel clerk, rental car company, bell hops, and neighbors. It's not depending on the government to keep us safe because they can't. Government is too big and too slow to respond to a ever changing threat landscape. Had we not spent the last five years alienating the muslim and mid-eastern communities in this country and abusing the few Arab allies we have in the mid-east, we might have been able to develop a community network that would have been effective and inexpensive (in relative terms).
No one seriously believes oceans can defend us, just like no one can seriously believe all the invasive technology being loosed on the people paying the bills is going to be any more effective.
It's all really quite insane.
That's our life, the big wheel of shit. - The Fat Man, Blue Tango Salvage
The most effective way of stopping terrorists is taking away their cause. Believe it or not, terrorists don't blow up hundreds of people as well as themselves because they "hate freedom" or any of that rubbish.
America's data set on terrorism is in the single digits, and the data they do have is only partially complete. This means the only system that can be programmed is a set of user-created rules that "flag" questionable behavior. The solution is a poor one and will only improve our chances at detection by a fraction of a percent. (Seems a huge price to pay for privacy trampling to me.)
In order to detect terrorism on American soil effectively, we'd need a larger data set. Otherwise we're just attempting to reverse engineer a process that essentially defines itself as dynamic enough to avoid detection. We'd need a frequent source of terrorism that we could derive models and nets off of. The immediate source that comes to mind is Iraq. If I were in charge of the NSA program, I think the best course of action would be to harness the call-traffic (satellite and domestic), email activity and other "data" that precedes suicide bombers (or other known acts of terrorism) in Iraq. Using this data you could train a system to recognize similarities in America. Short of that, anything the NSA is trying is a crap shoot.
No. Freeing up lines of communication, preparing quick and actionable responses to warnings, and better general population awareness are probably more effective than grabbing a billion pieces of data and sifting through it for answers. It's impossible for a human to know what to look for, and until the NSA comes clean in what it's actualy doing, there's no justification for stomping out the few freedoms we still have. There are better alternatives out there that can be done with the help of the community and still preserve the integrity of our privacy.
Armed with details of billions of telephone calls, the National Security Agency used phone records linked to the Sept. 11, 2001 attacks to create a template of how phone activity among terrorists looks, say current and former intelligence officials who were briefed about the program. (from the USA Today article)
Are they admitting to collecting details on domestic phone calls _before_ 9/11?
The masses are the crack whores of religion.
Can data mining identify terrorists?
No.
But it can identify people with large extended families who have relatives overseas and get an important call about a death in the family, notify all their North American relatives, and then have government agents show up on their door.
Every single pattern-based terrorist screening method I have heard about sounds like something dreamed up in an air-conditioned office by some dork who never gets out very much and thinks all people are basically like him (and anyone who isn't ought to be subject to government investigation.)
Hanging around public buildings taking pictures? Must be a terrorist. As opposed to say, just interested in taking pictures of public buildings because modern-day monumental architecture happens to turn you on.
Want to learn to fly a 747 but don't have any interest in a career as a pilot? Must be a terrorist. Unless you happen to be fascinated by aircraft and think that a few weeks of flight school would give you bragging rights to die for at your local RC club.
Like to pay with cash, even for purchases in the thousands like furniture or maybe a car? Must be a terrorist. Or maybe you don't qualify for a chequing account, or are just a little bit paranoid, or just don't fucking feel like doing anything else.
These sorts of unvalidated, non-empirical, "feels like the right thing to me", ad hoc, imaginary "patterns of suspicious activity" are a major threat to freedom because they demonize and may even criminalize deviancy from the norm. It is a characteristic of unfree societies that deviancy from the norm is not just looked at asscance by the majority of the population, but is viewed as grounds for suspicion of the most heinious acts.
Furthermore, such datamining solutions are not able to identify terrorists reliably even when they have all kinds of intelligence data entered into them. A report on the chilling-named MATRIX system indicates that the system was only able to identify 5 of the original 9/11 hijackers in a retrospective test, a 75% false negative rate, and it further identifed 120,000 other Americans who had a "high terrorism factor." Supposedly "scores of arrests" resulted from that list, although no one knows what the arrests were for or how many of those were sucessfully prosecuted. The odds are most of them were for drug possession charges that were laid as a result of the increased scrutiny certain individuals got by virtue of wholey baseless suspicions of terrorism. But let us grant 60 successful prosecutions for terrorist-related activities. That's a false positive rate of over 99.9%
And that was when the system was loaded with specific intelligence data, which is no longer the case.
Given the complete failure of such systems to detect terrorists in retrospective studies, and the horrifically high false positive rate, and the chilling effect such programs have on the freedom to be different, it is very hard to believe that their real purpose is to spy on Americans and impose a high degree of conformity on American society.
Blasphemy is a human right. Blasphemophobia kills.
Bah, who needs to be anonymous? I'll freely admit that I start off random conversations with "in" friends with, instead of "hello", "kill the president". Then I randomly throw in other potential keywords at random points later if I feel like it.
;)
Hey, if you hear of someone from Iowa ending up in Guantanamo, you know what happened.
As it says in the Constitution, Lenin is in my shower.
I don't think the question should be is it legal?
The question should be is it consistent with America's values? Or is it moral? And I think the answer is a resounding NO!
The problem when you ask about legality is that you get legal opinions with obscure analysis that circumvents the broader question of whether America SHOULD do this.
It's alot like the debate surrounding our system of legalized bribery (except we call it lobbying). "Oh, they paid for a plane trip, let's make those illegal." The debates center around the legal technicalities, but largely ignore the larger problem of targeted contributions directly affecting specific votes and the immoral culture of lobbying.
If we assume that the people at the NSA and other spy agencies are smart enough to know this too, then one has to ask what are they really trying to do.
The answer is that monitoring known actors (such as political dissidents) who are members of known groups works well with these techniques.
Here's my little essay on the subject (with some historical examples thrown in): http://robertdfeinman.com/society/surveillance_vs_ liberty.html
The bottom line is that secret police functions rapidly become tools for suppression of domestic dissent.
-- Robert D Feinman Landscapes, Panoramas, Photoshop Tips and Musings on Society
Not that I'm at all happy about the monitoring, but in fairness, would the NSA/FBI report massive success with the data mining? Doing so would inform terrorists (drug dealers, lesbians, Democrats) that the simple pattern of their phone calls can identify them, forcing them to change their methods of communications, undermining the success of the program. It might be sufficient for them to publicly leak stories that the program isn't working while reporting to the government that it's actually quite successful. It certainly wouldn't be the first time disinformation has been used.
An interesting aside: as reported by Bruce Schneier, al Qaeda members avoid Echelon by using shared Hotmail accounts. Rather than sending email, they create drafts and save them, and have a running conversation in the draft before deleting it. Not sending the email means the email doesn't trigger midpoint monitoring. Would they be doing that if they didn't know about Echelon?
Anyone who loves or hates any language, platform, or manufacturer, doesn't know what they're talking about.
Why do I get the feeling this entire thread was written by one person?
"forwarded to the FBI for investigation"
That dodge is how Bush can appear on TV saying "this NSA program doesn't listen to your calls", because they forward your calls to another program, at the FBI (and probably elsewhere). Feel safer?
--
make install -not war
There is a difference in that one is expressly and well-established to be unconstitutional, and the other is merely of dubious constitutionality and prohibited by statute (or, at least, the telcos turning over the information si generally prohibited by statute.)
OTOH, they are both the same in that they involve the gathering of information in which individuals have a legitimate, and recognized-in-law expectation of privacy, and therefore should not be done by the government in a free country except with a showing that there is some credible reason to expect evidence to be uncovered by the examination of the information associated with a particular target.
There is an easy solution which was known by our founders -- to intrude into the private information of a citizen, the executive takes specific information justifying the particular inquiry to a court, and gets a warrant if indeed that information shows probable cause.
A not-unlikely scenario:
1) A Pakistani developer starts an interesting FOSS project.
2) I test a copy and like it. He then calls me or I call him for a phone interview.
3) My next step is to call a bunch of sources in the U.S. and elsewhere, ask what they think of the software.
So with no family or friends in Pakistan, I am suddenly a potential terrorist threat by NSA standards. Uh huh.
It doesn't need to be a story about software, either. One about anti-terrorism activities could generate a similar call pattern.
On the other hand, I suppose that by current U.S. government standards, any journalist who makes a lot of calls to verify a story, instead of being a Good Little Boy and sticking to "official sources," is nearly as dangerous as a terrorist, anyway.
(sigh)
It's amusing to me that this whole program is self-defeating, even if there was any chance that it could work in the first place.
You just can't gather that much data from that many sources and not expect that someone will find out. Once the well organized terrorists know that the data is being gathered then they'll simply change their calling habits. These are smart folks, they'll figure out ways to obfuscate their calling patterns (use internet methods, call from payphones and hotel rooms, make only local calls, route calls through non-cooperating foreign phone networks using e.g. 3-way calling, etc).
But the government will still have the data and the only people left vulnerable to the database will be non-terrorists.
The smart people at the NSA must have known this when they designed the program.
Similarly with blog comments... a lot of it looks like spam, but it could be disguised commands, and it can be seen by people using search engines so there's a disconnect (cutout) between the poster and the recipient. All the reader would have to do would be to search on an innocent phrase agreed between the poster and the recipient and then view the cache of the page that matches that content...
they could be using Slashdot right now to coordinate the next big one...
Donald 'Duck' Dunn: We had a band powerful enough to turn goat piss into gasoline.
Secret database??? They snatched records from the phone company.
Calls coming into the country from Pakistan, Afghanistan or the Middle East, for example, are flagged by NSA computers if they are followed by a flood of calls from the number that received the call to other U.S. numbers.
So if one of our servicepersons calls his/her worried mother to reassure her that he/she is ok after a particularly hostile engagement and she wants to let the rest of the family know, which one is the terrorist?
The spy agency then checks the numbers against databases of phone numbers linked to terrorism, the officials say. Those include numbers found during searches of computers or cellphones that belonged to terrorists.
If they find something suspicious, they check to see if it's connected to a known terrorist phone number? If they already have a list of known terrorist phone numbers, then just what the hell are they looking for?
"Able Danger" identified Atta and three of the other hijackers pre-9/11.
http://en.wikipedia.org/wiki/Able_Danger
Instead of the government trying to cover up the success of Able Danger, it should be initiating twenty or so Able Danger-like data mining programs.
That that is is that that that that is not is not.
Bruce Schneier wrote an interesting piece on why data-mining not only doesn't work, but can't work in the context of finding terrorist plots:
t w=wn_story
http://www.wired.com/news/columns/0,70357-0.html?
In a nutshell, his premise is that the underlying assumptions that make data mining work for such things as credit card fraud don't hold when searching for terrorist plots. Also, that trying to apply those models will result in a flurry of false negatives so large as to make the whole effort useless and a waste of resources which could otherwise be better spent. It's hard to argue with...
----- My opinions are my own, etc, etc.
"If I'm sitting outside of a gas station at 1:30 am with a ski mask on and a cop pulls up, no crime has been committed but the officer would have probable cause to stop me and search my car because I'm behaving in a suspicious manner and the officer could reasonably speculate that a crime was about to be committed or had been committed."
Not at all. I think you need to research your rights better. The cop could stop you, he could ask to search your car, but would not be allowed to search it without a warrant unless you gave him permission -- same with your personal effects (like what's in your pockets). If he smelled marijuana or gunpowder residue, or saw blood, then he'd have grounds. But it's absolutely scary to me that people would believe that wearing a ski mask is grounds to be searched.
If you're not even aware of your rights, how do you know when they are taken from you?
"Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
The fact that there hasn't been another attack doesn't really prove anything more than the fact that there was no attack for three years prior to 9/11, or five years prior to that. If you could show that the number of terrorist attacks per unit time under the current security policy are lower than the number per unit time under our old policies, then you'd have a case. But just saying "there hasn't been any attacks for five years" doesn't mean anything --- it could simply mean that terrorist attacks are rare regardless of your policy.
Of course, there is also the "we're at war" aspect. Why should terrorists go to the trouble of trying to kill Americans in the US when there are a whole bunch of Americans in Iraq that are much easier to target. You could just as easily argue that the lack of attacks over the last several years is due not to better security policies at home, but the fact that terrorists are occupied killing Americans abroad.
On the other hand, you have some fairly strong evidence to suggest that our current security policy really isn't any more secure than it used to be. Just last month, auditors tried to sneak weapons onboard airplanes, and succeeded in the vast majority of attempts, despite consciously making the weapons easy to discover. At the same time, you had the Israeli guys audit our airplane security, and conclude that it was "not so much a system for protecting Americans as it was a system for annoying them."
A deep unwavering belief is a sure sign you're missing something...