Slashdot Mirror
Former FBI Agent: All Digital Communications Stored By US Gov't
New submitter davesays writes "CNN anchors Erin Burnett and Carol Costello have interviewed Former FBI Counterterrorisim specialist Tim Clemente. In the interviews he asserts that all digital communications are recorded and stored. Clemente: 'No, welcome to America. All of that stuff is being captured as we speak whether we know it or like it or not.' 'All of that stuff' — meaning every telephone conversation Americans have with one another on U.S. soil, with or without a search warrant — 'is being captured as we speak.' 'No digital communication is secure,' by which he means not that any communication is susceptible to government interception as it happens (although that is true), but far beyond that: all digital communications — meaning telephone calls, emails, online chats and the like — are automatically recorded and stored and accessible to the government after the fact. To describe that is to define what a ubiquitous, limitless Surveillance State is."
I doubt they have the storage capacity.
Table-ized A.I.
Limitless but also almost useless currently. The point is to wait for years and then crack today's encryption with tomorrow's computers.
...but really this just seems to be par for the course. In fact I'm thinking he's really just confirmed what we all knew deep down.
Someone correct me. It just doesn't seem feasible.
I'm sure what he's saying is true, in a very broad sense. Cell phone conversations, texts, and major/popular version of things like video chat (skype), IM (yahoo messenger), and general social media (facebook, twitter, etc).
This guy seems to be implying that the government has some kind of man-in-middle technology that intercepts and records *all* traffic, which simply isn't true. Unfortunately, either he or the news agency is trying to paint the whole thing as just that.
I have done digital communication that never entered the internet. This includes LANs, corporate networks, the USB wire on my desk, the IPC pipe between 2 applications, etc. I've even send the old finger binary number 4 to someone optically. If they could record all these things, there would have to be a secret very high bandwidth network in-place connecting every digital device, which seems impractical, or even impossible.
Now, if you mean all traffic over the internet-backbone in the United States, sure, they could do that. Time for encryption everyone. At least Google likes to pipe their traffic though the corporate network around the internet backbone, so that needs a different approach to log.
The plaintext era is over.
Either this is true and so secret even most of law enforcement doesn't have access or it simply isn't true. Having run a large enough telecom operation to deal with CALEA I can say for sure that law enforcement very much needed our help to do anything with our customers' communications. Not only did they need to come to us with proper warrants in the first place, but they barely had enough technology sense to be able to do anything with it. Anything more complicated than taps and CDRs never even came up.
Clemente: 'No, welcome to America. All of that stuff is being captured as we speak whether we know it or like it or not.' 'All of that stuff' â" meaning every telephone conversation Americans have with one another on U.S. soil, with or without a search warrant â" 'is being captured as we speak.' 'No digital communication is secure,' by which he means not that any communication is susceptible to government interception as it happens (although that is true), but far beyond that: all digital communications â" meaning telephone calls, emails, online chats and the like â" are automatically recorded and stored and accessible to the government after the fact. To describe that is to define what a ubiquitous, limitless Surveillance State is."
Dear US official;
All of my communications are sent via encrypted proxy, and set to stream constantly. The proxy dumps into Tor and a half-dozen other networks. I originally did this for shits and giggles, to see how hard it would be. I will admit the latency is a bit higher than doing it locally, but it is very usable in spite of this. I also signed up for the Tor Cloud project and run an EC2 micro-instance to help others do the same.
Originally, I just did this as an experiment, but after reading things like what you're saying and realizing that we've become a surveillance state on par with Iran, China, and North Korea (where did they get their filtering and monitoring hardware from again? Oh right: We gave it to them), I decided to keep it.
I don't do anything special with my super duper encrypted "all the things" setup. I wish I could say I was some elite ninja hacker or something, but all I really do is browse internet forum sites and read the BBC news, and you know, download a few TV shows here and there. I'm one of those people that doesn't have anything to hide per-se, but when I live under the tyranny of a government that has turned their citizens into the enemy -- the attitude that we're all criminals or potential criminals, and must be monitored pre-emptively, I feel like it's my duty to frustrate the hell out of people like you.
So I have been helping friends, family, and strangers, set their computers up the same way. Yeah, I know, some of them will probably use their newfound freedom and anonymity for evil, but frankly, even a terrorist attack a week and all the rantings in the world from you (that may even be justified) about how criminals can use this technology for their own nefarious purposes, doesn't deter me.
You crossed a line; Morally, ethically, constitutionally. By criminalizing the average citizen, you have become a bigger danger than all the terrorists, all the "real" criminals. You are corrupt, dangerous, and seek to undermine our democratic way of life. You hide in the shadows and see conspiracies everywhere, and are convinced of your own righteous cause. You are as dangerous as a religious fundamentalist, because just like their dogmas, yours demands absolute purity. There will always be more justifications to invade the privacy of others.
So I will continue to teach anyone who wants to, how to fight back against your tyranny. You're a threat to the way of life of not just myself, but my peers. You're a danger to all Americans -- you view us as the enemy. Your own people.
You've lost your way.
#fuckbeta #iamslashdot #dicemustdie
...I was wasting my life watching The X Files.
WRONG!
Scully alone made it all worth it.
Aaaaaahhhh... Scully...
This is probably the most telling bit of it:
CLEMENTE: We certainly have ways in national security investigations to find out exactly what was said in that conversation. It's not necessarily something that the FBI is going to want to present in court, but it may help lead the investigation and/or lead to questioning of her. We certainly can find that out.
Basically, this capability exists, can and does get used, BUT the fruits of which aren't directly presented in the courts, because that would divulge too much as to its existence. Instead, it gets used to get the suspect to admit what might otherwise be unattainable through a normal interrogation.
Now the scary part:
This could probably directly provide evidence for not just the Boston Marathon case, but many many other criminal cases in this country right now. For all those other cases though, we risk not convicting a criminal, or worse wrongfully convicting innocent people.
It's kind of sad/scary to think that the FBI effectively has a digital oracle that could provide the information to make many trials look like daytime soap operas.
http://en.m.wikipedia.org/wiki/Utah_Data_Center. This place comes to mind even if possible this place has tons of storage
Might makes right.
Protip: If you want actual freedom, you need to gain power. And to gain power, you need to align yourself with the powerful.
The question for you is: Which members of our society are more powerful: the socialist statist that aligns themselves with government? or the gun-owning libertarian that doesn't want to be tread on?
Figuring out who is power, and subsequently aligning yourself with that power, is going to be key to your success. Feel free to argue against that.
so someone at nsa proved p = np... or at least how to factor large numbers.
That in that case classification is done more or less automatic.
Down to the individual, we may assume.
And yet, they are unable to prevent [insert latest bombing or amok event here].
Clearly a government gone anal.
Only aliens would have the storage technology.
Table-ized A.I.
No one else would post such obvious crap.
It is impossible for the government to record all digital communications because they aren't privy to them. Unless you mean to tell me they have a back door in my open source mail servers which communicate on their own private networks within my organization over physically secure channels.
They don't ever get access to a lot of digital communications so they can not possibly be recording it all.
typical timothy story, so blindly false, yet somehow the moron keeps posting shit.
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
Pshaw. I've got 26 fault-tolerant terabytes just for my media. Every piece of hardware is available from regular vendors like newegg, amazon, microcenter, etc. It fits in a mid-tower case with room to spare and the capacity could be doubled by switching to 4tb drives.
Google was expected to pass an exabyte of data years ago. Amazon's somewhere around that range with their cloud services. Facebook claims a petabyte of duckface pics and videos. And those are companies that are designed to be making a profit. Government agencies don't have to worry about things like profit.
The question isn't whether it's feasible to store that much data but whether the government is capable of managing the creation of such a data center. They don't have a great record when it comes to IT projects. At least not public-facing ones.
see subject
rewriting history since 2109
I doubt this, not because I believe the goodness of the government as such, but because of the logistical impracticalities it would entail. Think of the sheer amount of storage, electricity, infrastructure, personal, computing resources and so on that you would need in order to perform this feat. The numbers would be boggling and would account for a significant portion of the worldwide sales of all hard drives, tape back ups etc, etc.
You would then have to work with the absolute enormous amount of data in a usable manner which as anyone who has ever worked with very large data sets knows is easier said than done. When you have this much data it's a little more complicated than running a few SQL queries against a given person. The sheer volume of data would make this entirely unusable even if they could pull it altogether.
You would also need personnel from IT types to human resources and so on. This would be one of the largest projects in the country and would have a noticeable impact on unemployment. Physically, where would you put this and get the electricity to run it all? Where would you get all the people with clearances? The logistical realities make this a non-event for domestic communications, it just isn't possible. I'm not even talking about the largest wholesale violation of the Constitution in history if this were true. Sorry, but this doesn't pass the sniff test.
...I think maybe the militia nutjobs are the most sane of the citizens...
How you use language, what you say, etc. can be used to identify you and/or the region you come from. You may use some phrase only common to a smaller region than you might imagine. You might mention the details to a lake that isn't as ordinary as you would guess. These are but examples.
I know I for one, a few seemingly ordinary details of a nearby park and my age would likely be enough to find me. Can you guess one feature of that park that I have, I promise you, mistakenly given above? How much more do I need to say to give it all away?
If not you, how many of your friends or others you've helped will say too much? About themselves or you. Have you told your friends and others that their phones are geotagging their photos?
As for the government reading this, my only question to them is this, "Where's my stinking refund!?"
and here I thought USA was the country of "freedom and democracy" and was so much better than China??? Was I lied to then????
Cryptome got the Aaron Swartz PACER file: http://cryptome.org/2013/05/fbi-swartz-pacer.pdf
On Page 181 you can see they intercepted his home phone without a warrant but weren't able to obtain the content of his other phone calls because his number wasn't in some classified system.
Apple has built multiple massive data centers throughout the country to support their iCloud service, at a cost of hundreds of millions of dollars, and using many gigawatts of electricity. Compared to ALL of the digital traffic in the US in a single day, iCloud is but a drop in the bucket. The resources required to store all of that data would be ridiculously ENORMOUS, especially when you consider that you'll also need redundant data storage and backups. It would require dozens of facilities running 24/7, each with it's own direct "backbone" connection, huge amounts of power and cooling capacity, backup generators and cost billions of dollars every single year. You would need dozens of terabytes of storage to store just one day's worth of data. The amount of storage required to store just one year's worth of communications would be astronomical.
After all of that, you would need to consider a method of cataloging and indexing all of that data, and a way to search and recall all of it in a reasonable amount of time. Were we in the age of quantum computing, I could see all of this as being possible, but with today's technology it would be a practical impossibility.
They do have good off line secret installations. Where security is really important. I went to Ft Huachuca for computer security training and they did a fairly decent job for low level IT staff there. The instructors and some of the other people at that base genuinely knew what they were doing.
DoD care a lot less about every day logistics systems for barracks assignments then they do about keeping under wraps their mission planning or god knows what else, I couldn't get anything out of anyone that mattered.
I'm no longer serving. But you can rest assured there are some in the service that are good at what they do.
He's talking about Carnivore isn't he?
Yay me!
We where trying the calculations today could be a little off though:
http://www.deadzones.com/2011/05/how-many-cell-phone-calls-are-made-day.html#.UYZ6K_FAZxQ
according to that 6 billion calls daily
http://www.theregister.co.uk/2013/01/30/mobile_phone_calls_shorter/
1 minute 40 seconds per call
people talk about 150 words per minute
http://answers.yahoo.com/question/index?qid=20080526032554AAB28AF
avg word length 5 characters
words in avg conversation 1.666666 (which is 1 min 40 sec) * 150
100
avg conversation has 250 words * 5 letters
1250 +249 for spaces
that's 1499 bytes per conversation uncompressed and not caring about punctuation and line breaks
so each day it's 8994000000000 bytes of data
1499 * 6000000000
8.18 tb day
using something like kgb compression 2.5/800
0.003125
compression rate 3.2GB
1.14062 TB yearly
This is of course assuming they are converting the speech to text.
So how do I access my data?
I arranged to meet with Lionel this Friday but I can't remember the name of the bar, and now I lost his cell phone number!
Awesome service by the way, thanks FBI!
Hej! Nasi tu byli!
Following where pulled from sites via search:
6 billion calls daily @ 1 minute 40 seconds per call
people talk about 150 words per minute, avg word length 5 characters
words in avg conversation 1.666666 (which is 1 min 40 sec) * 150
100
avg conversation has 250 words * 5 letters
1250 +249 for spaces
that's 1499 bytes per conversation uncompressed and not caring about punctuation and line breaks
so each day it's 8994000000000 bytes of data
1499 * 6000000000
8.18 tb day
using something like kgb compression 2.5/800 based off the office example
0.003125
compression rate 3.2GB
1.14062 TB yearly
Assuming they are converting voice to text.
Could I ask them to restore that email I accidentally deleted last week?
I work for a moderate sized phone company. Customers in the millions. The government has no link into our systems. WE can't even record all of your calls and monitor all your internet traffic. Think about it this way, your ISP likely doesn't even have enough bandwidth to provide you with the speed you're paying for some of the times... Netflix on a friday night for example. Do you really think they have the extra bandwidth to ship all that data off to the government as well? Phone calls are a whole other animal, and are mostly still analog. Duplicating that would involve upgrading the switch... an at least 30yr old piece of obsolete equipment... It just doesn't make sense. Sure, the government could pay for all this stuff... but it would be a HUGE project. Everyone in the company would know. The equipment in our data center is very obvious... we all know what each piece does. There's no mysterious black box in the corner... and there's no way they could be tracking everything without us knowing. There would be at least 1 piece of weird equipment somewhere. I've neither seen nor heard of any such equipment. On top of that, all that data would be meaningless without access to our databases. Capturing the data or phone calls raw would just give you a mac address or phone number. You wouldn't know who was using those numbers. So you'd have to query our database... a database that changes regularly... new systems come online all of the time. So they'd have to have access from outside of the company, so holes in our firewall, make SOAP requests into our system, Have an active user account, make requests to dozens of different DBs, hundreds of Tables, know how all their joins work, know when system changes go in, and on and on... No such thing could happen without the entire company knowing about it. It's just not possible.
Oh, wait...
Exactly. If as the article claims, the government is archiving EVERYTHING then they are archiving all those duckfaces on Facebook, all your Amazon browsing, all your Netflix instant streaming, all your Box, Skydrive and iTunes syncs and every Skype video conversation. The Government would need Google + Amazon cloud + Facebook + Netflix + whatever else and it would have to grow it constantly, unless they had some nice de-duplication or filtering system which would then break the integrity of saving everything.
What a pity. I remember a time when Slashdot was filled with people who were smart enough to know that this guy's story is simply a load of crap and would dismiss it as such. Unfortunately, it seems the Slashdot crowd is now no smarter than the morons duped by regular news audiences.
Those who know can't talk.
Those who talk don't know..
Someone correct me. It just doesn't seem feasible.
Of course it's feasible. For each unique human soul, all they need is one bit ("evil"/"not evil"). Throw in some standard compression, and it probably all fits on a few 3-1/2" floppies or so.
As far as digital signalling ("digital communication") is concerned there is no relevant difference between loading a web page, sharing files, playing a game and a telephone call:
it is all binary signals encapsulated in packets.
A lot of it is encrypted uniquely, some of it is not. Deep packet inspection is relatively expensive and deep packet inspection of encrypted signals are at best a shoddy guessing game and for every assumption made to remove protocol overhead or network housekeeping or static "broadcasts" or "uninteresting" ports or whatever another loophole the size of the sum total of all voice communication is created.
So no everything is simply not being recorded and will never be, there simply isn't room, time, or capacity to record/duplicate, assess, and (possibly) store everything as it is being transmitted again and again in various forms even if the content might be the same (in fact all the content is the same by the nature of binary numbers: proper context is king and the only thing that differentiates lolcats from terrorist plans for nuclear genocide in a digital world made up of a constant stream of the same variations of numbers between 0 and 64 bytes combined to form anything between 0 and infinity).
This doesn't mean that a lot more than could be legally stored isn't stored --it could well be, maybe that's still the case useless (with regard to truth) as that would be-- however it is not "everything" and most likely completely and utterly useless unless specifically targeted.
The idea of storing all digital communication is nonsensical as the idea of downloading the internet.
FBI is not supposed to stand for Fucking Boneheaded Idiots, no wonder the guy is "former".
TL;DR: Meh!
There's no NDA that requires the FBI lie about what the NSA is up to. Just because they got immunity in 2007 doesn't mean they were granted the right to mass surveillance, its still a crime even if the punishment is gone.
You should read the article, and also read up on the massive data centers they've been building. If you're building a data center bigger than Facebook+Google's combined, then you're not doing targeted surveillance.
"Every day, collection systems at the National Security Agency intercept and store 1.7 billion e-mails, phone calls and other types of communications."
"It would also help explain the revelations of former NSA official William Binney, who resigned from the agency in protest over its systemic spying on the domestic communications of US citizens, that the US government has "assembled on the order of 20 trillion transactions about US citizens with other US citizens" (which counts only communications transactions and not financial and other transactions), and that "the data that's being assembled is about everybody. And from that data, then they can target anyone they want."
"Despite the extreme secrecy behind which these surveillance programs operate, there have been periodic reports of serious abuse. Two Democratic Senators, Ron Wyden and Mark Udall, have been warning for years that Americans would be "stunned" to learn what the US government is doing in terms of secret surveillance."
They don't have a great record when it comes to IT projects
True, if you discount the Internet, the modern computer interface and the electronic computer itself. Of course there is a pattern here - the projects were commissioned by the government (the military) but built and maintained by other parties, like universities and private companies. So is it possible for the government to simply fund a secret project like that and then ask one of its IT contractors to carry out the work? Not only possible, but almost certainly the case. I know there are companies who would be more than happy to make such a project happen.
....are they learning how pointless their job is? For certainly the mass majority of the population are of people just living their lives and should not be expense'd by the few looking for trouble.
Its impossible. They cant do even half of what he suggests they do.
From a logistical standpoint how do you constantly monitor, record, sort, log and store information on billions of people every single second of their life? I cant even begin to fathom the sheer amount of manpower it would take to do that, not to mention the money spent and how the hell do you store all of that info for longer than just a few seconds?
Emails? Phone calls? Messages? Online chats? Every single digital thing every single American citizen does every single second of their lives? That's just bullshit right there.
This guy is either so damn desperate for attention he will do anything, or he has just cracked and gone nuts. Probablly has a tinfoil helmet on right now because the FBI is recording his brainwaves to store in a facility on mars because the rover up there is a secret government device meant to blow up the sun. Oh and I bet he thinks underwear gnomes come into his house at night to steal his boxers so they can make a huge profit.
Cellphone voice communication often uses bitrates of 8kbps. We'll be generous and say it's 10kbps 10kbps x 60 seconds x 60 minutes / 8bitsperbyte gives us 4.5MB of voice per hour. If you talk 8 hours a day 365 days a year that's a little over 13GB of data, and with current storage pricing it's less than $1US worth of hard drive space per year. And that assumes they are storing everything forever -- a modern processor can handle a *lot* streams of voice-to-text in real time, so you could keyword search for "interesting" words, and throw away any captures that don't seem worthwhile.
The calculations for email are even worse -- most people probably generate less than 1GB per year, and the text is likely only a fraction of that. I suspect that twitter and facebook and other social media posts are even shorter and thus require even less storage for the textual portions of the messages. The images may be larger, but you only need need to store the outgoing ones.
Storage doesn't seem like a huge financial or technical barrier. Getting access to the data to begin with is a larger challenge -- you need to put some substantial gear on some fat pipes in the telcos. The NSA Narus box in the San Francisco ATT central office made the news a few years back, a more capable system would likely be noticed.
Sorry but he is bold faced blatently lying. There is now way that my digital comms channel between offices is "recorded" it's on a private point to point T1 circuits. Let alone the sheer volume of data. One of my clients is on the Internet II backbone and they transfer 22TB of data every hour. So where is the US government getting all the storage for even the content flying about from university to university which is a drop in the bocket compared to what the Public Internet is sending all over right now per hour. You are telling me the Govt has a copy of the video I shot with my AR drone this afternoon? Where was the black van that was intercepting the signal?
It's a guy that doesn't know jack spewing made up crap to justify his new pundit job at CNN.
Do not look at laser with remaining good eye.
cell phone companies provide CALERA and other hooks for federal and local monitoring. all the voice communication is digital; your 30 yr old analog switch must be landline only. as other posters have said, the question is how they filter the available data and store meaningful connections to subscribers. Every cellular provider can track by switch (geographic area) or by individual subscriber number. It's more likely that conversations are scanned for significant words, and that subscribers of interest can be collected along with everyone who communicates with them. The "weird equipment" is access from police, etc to run commands on the cellular network devices. The monitoring commands are usually reserved for highest level access, and are typically not documented in operator manuals.
On the internet, 640 terabytes of data is transferred every minute. That means in a year, you have 330 exabytes of data. Not only that, you need the infrastructure to transfer it. You can deduplicate and stuff, but even deduplicating that much data is not exactly an afternoon hack.
Think of that: you're adding 640 terabytes to your database every minute.
"First they came for the slanderers and i said nothing."
The amount of storage would be phenomenal, but the ability to process and do something with it, would be impossible.
Round the living population up to 8 Billion. At one bit per person, not including lookup tables and such, it seems to me that it would amount to about one billion bytes or about one GB. Just a bit more than a few floppies...
Ever notice that Cobra Commander sounds an awful lot like Star scream?
There was a Frontline a while back about this: http://www.pbs.org/wgbh/pages/frontline/homefront/view/.
The EFF notes similar here: https://www.eff.org/nsa-spying
Neither stated that the government is recording _everything_, though.
As I recall from the Frontline documentary, the NSA argues that sweeping up and recording emails and conversations does not require a warrant if they do not actually read or listen to the recorded information. They say they would need a warrant for wire tapping if they want to open up a record and examine it.
The burning limo and 5 dead on the San Mateo Bridge were having a mobile Crack party when the mobile lab blew up.
This is much the same as the large meth lab blowups at the Boston Marathon; those were being operated by Boston Police.
why he's a former FBI agent. Somebody finally realized he's full of crap an a liability.
Historically, whatever tech is in the public view, there's usually much more advanced stuff that's classified. Considering the U.S. government has been pounding the anti-terrorism drums for a dozen years now, it's not beyond reason to think they've developed some incredible stuff we have no clue about. Years ago, they already had a lot of this capability. There's also Duqu/Stuxnet, which the public only found out about after it was active for a few years. So I'm sure they've found enough storage space, perhaps even using companies like Facebook & Google to help them keep it. The real question is, what's their reaction time? How quickly is the gov't capable of responding to what's being said digitally. There's a ton of data every day, but it doesn't matter if you can cache it all if it'd still take you a couple days to detect what you need/want to find in it and react to it.
Doing some rough estimation here, based on Googling various averages and picking reasonable-seeming results. Let's assume they're storing phone conversations and web traffic.
I found a figure claiming that the average American spends about 23 days/year on the phone. Assuming 16-bit audio at 16kHz (generous for phone audio), that comes to about 64 GB/year/person. Multiply that by 300 million people to get roughly 20 exabytes (20x10^18 bytes) per year. Do some simple processing (removing silence, compression, etc) and that figure will drop significantly.
Cisco says that global internet traffic is on track to reach about 1 zettabyte (10^21 bytes) exabytes per year. Eliminate commercial streaming services and routine binary downloads and that figure will probably drop significantly as well.
According to Wired, the NSA's new facility in Utah will be equipped to handle data on the order of yottabytes. (10^24 bytes). 1 yottabyte * 1year/zettabyte = 1000 years. That's a millennium's worth of storage at current rates.
Internet traffic is approaching 1 zettabyte/year. Ignore the extra "exabytes."
They should treat it as a free cloud backup service, and offer everybody back their old communications when requested.
who cares - as long as facebook doesn't get access to it...
The only time we record anything is when there is an official Calea request. Otherwise, standard POTS telephone calls are unmonitored (at least as far as local/local calls are concerned).
Roughly, it would cost them 10 million dollars per year, assuming a cost of $50 per 2TB HDD and an average talk time of 1 hour a day, for a population of 200 million. Codec works at 2KB/s
It's cheap. It's chump change for the FBI. Of course you need additional equipment, but storage-wise it's extremely cheap. Please someone correct me if I'm wrong.
Yay for rush transcripts and for the Guardian forwarding it on without verifying who said what. The CNN transcript got most of the words that were spoken, but misattributed them. The line in the summary was actually Geragos, not Clemente, although Clemente was nodding his head in agreement as he spoke. Here's a crappy YouTube clip.
A corrected version:
BURNETT: And Tim, is there any way, you know, obviously, there's a voice mail they could try to get the phone companies to give that out at this point. But if it's not a voice mail, it's just a conversation, there's no way they actually can find out what happened, right, unless she tells them?
CLEMENTE (former FBI counter-terrorism expert): No, there is a way. We certainly have ways in national security investigations to find out exactly what was said in that conversation. It's not necessarily something that the FBI is going to want to present in court, but it may help lead the investigation and/or lead to questioning of her.
BURNETT: So somewhere it's being digitized--
CLEMENTE: --So we can certainly can find that out.
BURNETT: -- or they can actually get that? They were -- people were saying, look, that would be impossible -- that's pretty incredible, what you're saying.
MARK GERAGOS (criminal defense attorney, author of "Mistrial"): No, welcome to America. All of that stuff is being captured as we speak, whether we know it or like it or not. [Clemente nodding his head]
BURNETT: Note to self, as Deb Feyerick just said, yes. All right, thanks very much to both of you. Obviously, that right there is a very significant thing, because people have been saying if there's a conversation, if it wasn't a voicemail, they don't know. If they can find out, that could obviously become crucial.
America hasn't got the money to do stuff like that!
"They do have good off line secret installations. "
BS, their huge data centers are known, if you spend billions on a data center, people notice, the latest is in Utah, and its bigger than FB's:
http://www.wired.com/threatlevel/2012/03/ff_nsadatacenter/
There's nothing secret about it, and if Google can store your emails, voice, messages, and billions of traffic, adverts, and a copy of the web for its search engine, Then the NSA can also do it quite easily for just the email/voice/message part.
I mostly agree with your post, but not the secret part. They're doing in plain view and people are in such denial about it.
and you just both proved how stupid Americans are.
BOTH SIDES OF YOUR PARTIES ARE BRIBED OUT TRAITORS to the corporations whom supply your armies and cops....
THUS they have a stake in bribing your rights away....
that you do not see it , that Americans avg math rank is 31st in world rankings is PROOF you have what you deserve.
Have gnu, will travel.
its common among hackers to NOT use tor for your privacy and security....
im sick a trying to tel yankies anything why educate a populace that has a system that is run by corporations that do the rights taking any ways.
YOUR DOOMED
its over a long time ago.
oh heres a tidbit you dont know who ever i am ...i know of every single ip address the fbi has out there ...and they have millions of honepots
in 99 it was about 65 million, and keep on thinking that they are idiots...they know people like that above poster will yack so they will allow you to think they are stupid.
Anyone with a lick of common sense and a smattering of technical understanding already assumed that this was true.
it is perfectly reasonable for hundreds of people to know about something and keep their mouths shut, because if they say anything they could face decades in prison.
if you read James Bamford's books, you will begin to realize that most of the major US computer companies, from Cray to IBM, were propped up directly and secretly by the NSA to build supercomputers for it, secretly, years before the technology would reach the public.
of currently known, publically acknowledge information technology. They have actually been secretly behind the development of most supercomputers in the 20th century.
read James Bamford's books. they probably have a backend super-fiber into iCloud, amazon, google, etc etc etc.
i know that you work some place and thats impressive but there are just mounds of evidence that prove you are wrong. and several other comment threads above with the same idea.
"i work for a telco, there is no way this is possible"
"ok thats great, except for several well known court cases that prove the exact opposite of what you are saying"
and a few other cases essentially prove youa re wrong.
there is a mysterious government fiber out port. and a mysterious government room where the fiber goes. there are many of them.
you have no way of knowing what configuration they are using.
please read jim bamford's books.
they dont need to actually have it. thats the nice thing about being the government. you just print more money.
now, some worry warts will point out that eventually this practice will devalue your currency and collapse your economy through hyperinflation, as has happened in several countries over the past 5000 years of civilization, repeatedly.
but hey. we are the USA. we are immune from the natural laws of stuff.
Your switch passed the call to another, and in so doing, routed through their sniffer.
Especially if you called on an AT&T Cell phone.
I worked on voice recording systems. Your comment is false. If we could record every message into a banks call center with a few PCs and a few rhetorex cards more than a decade ago, we can certainly record pre-compressed data on modern kit. The biggest limit we had then was the rhetorex cards ability to digitize/compress the data, if the data came through the switch compressed (as it does now), that would have been a trivial routing problem.
A switch in the exchange IS just a computer, it routes voice packet data, it does it successfully and without a capacity issue. If a switch can route voice, storing the packets is a trivial issue now.
Email would be trivial (Gmail records it, Yahoo records it, so of course they can record email).
"your ISP likely doesn't even have enough bandwidth to provide"
Not the same thing, a million people visiting a website, is a million entries in a log file, not a million *copies* of the data on that website. Again I'd point you to Google Analytics which does exactly that for Google. It logs your visit, and breaks the data down for the website owner (and for their advertising system).
You're saying *impossible* to things THAT ARE ALREADY DONE AND FREELY AVAILABLE!
Narus is just one of many deep packet inspection company
http://www.narus.com/solutions/narus-nsystem
So what did the Boston bomber say to his wife when he called her? They would be looking at their humongous HDD instead of following the terrorist's wife, or is the HDD corrupt? Well Clemente claims, and we exclaim.
You don't save it. You buffer it, and scan it in real time. If the computers "hear'
something that someone should check out, just persist the stuff in the buffer.
... random number exchanges with my friends?
now we need to go OSS in diesel cars
640 terabytes a minute ought to be enough for anyone.
...If they capture all the transmissions originating from their listening devices and ending at their storage device across the country (on top of storing the transmissions that they intercept.)
No, re-read his comments. He only mentions that they are recording all voice traffic, not all data. He goes on to say that all digital communications is insecure, but not that they're actively recording all data traffic.
Voice comms is very low in bitrate, and it hasn't scaled up exponentially like general internet traffic, so I have no doubt that the technical capability to do what he says exists. Whether they are actually doing it is a separate question.
all digital communications - meaning telephone calls, emails, online chats and the like - are automatically recorded and stored and accessible to the government after the fact
It is not possible to record a communication that does not pass by the router that you tap. It is hard to record the same if different packets are sent to ten different routers and take different paths to destination. It is not feasible to decrypt a communication that is secured with AES and a new OTP as a key for each block. It is not practical to detect a communication that is done in the zeroth bit of a large photo of a rose bush and posted on Flickr. There are other examples of communication that cannot be recorded or made accessible to anyone but the parties that communicate. Whoever is telling this tale, he is just trying to scare the proles into believing that watchers are omnipresent and omniscient. A new god, by the looks of it. Kneel before me, worms, NOW!
Oh you're just silly. We just number people the right way and it'll fit on a post-it. One readable without any magnification except what's in your eye. Suppose there are X evil people, and Y not evil people. We number everyone evil between 0 and X-1, and everyone not evil between X and X+Y-1. Problem solved. You only need to write down the value of X. No need to store any bits. The first X bits would be 0, the next Y bits would be 1. Easy-peasy.
A successful API design takes a mixture of software design and pedagogy.
Perhaps, but It wouldn't necessarily need to capture every single bit of data being transferred. The titles of the movies you stream from Netflix could be collected, but the stream itself ignored. Ditto for the eBook you bought, the MP3 album you downloaded, the game you pirated, etc. Or put another way: The only time your personally created video of your cat is collected by the feds is when you upload it to your site, YouTube, Facebook or whatever. Every time that it's liked, re-tweeted, emailed, hotlinked, or otherwise used elsewhere, It's more or less recorded as "At [TIME/DATE], mianne viewed phantomfive's cat video obtained at prior {TIME/DATE]." I have no way of knowing how much bandwidth/storage that would entail, but I would guess it'd be a much more modest scale from nnn MBs/min to nn GBs/min.
Javascript, cookies, flash, and ActiveX must be enabled in order to view this sig.
... is absolutely possible. That is what the company I work for has been doing for a while now. For commercial entities we market it as social network sentiment analysis.
The technology can tell you immediately how your brand is trending, analyzing twitter feeds, facebook, blogs, web searches etc.
It scales nicely. And yes, we have a very large footprint in DC.
This was actually part of why I left the US. As a consultant it seemed security clearance was becoming more and more important. Not the line of work I was interested in.
I don't understand how none of you realize what a service the government is doing for you all!
Who needs backups??
Data center burn down? Didn't get a backup?
One freedom of information request later and your back up and running.. Thanks Uncle Sam!
"On the internet, 640 terabytes of data [intel.com] is transferred every minute. "
A million people visiting a website, is just a million entries in a log file NOT a million copies of the website. Google Analytics already does this, Google adsense already does it. They both track who visited what website.
Do you imagine when a million people visit Netflix and download Inception, that the NSA keeps a million copies of Inception and dedupes that later??? No, they just log it.
For communications, well FB already stores all messages, Skype does it too, all email is there on Hotmail, Gmail, and Yahoomail, it's quite trivial to record it. Most telcos offer voice mail too.
Voice data is more difficult to store, you'd have to throw away some of it. But who called who when, that is already logged and stored by the telcos for billing and it's quite trivial to keep. I recall that data is quite small.
"Think of that: you're adding 640 terabytes to your database every minute."
Straw man, the *unique* data is already tracked by companies for various reasons and is nowhere near even a tiny fraction of that data. We know NSA gave itself a live tap into the network (and subsequently the telcos got immunity for it), do you imagine they didn't get a tap into the voicemail system too?! The call logs too? The billing records too?
If they grabbed the most difficult data (the live tap) they grabbed all the easy stuff too.
640 terabytes of data is transferred every minute.
And how much if you only record each youtube video once? . And log only meta information about vpn/ssl/https traffic. (source / dest / time...) And just ignore netflix, hulu, itunes, and the top 10,000 hollywood torrents...
is this the same government thats using reel to reel tape on 16 bit IBM mainframes running COBOL?
Tennessee is still advertising for AS/400 operators, California is still hiring cobol programmers, you fucks cant swap out the punch cards fast enough to store it all
Let me point out that slashdot links to news from the web and there is not that much original journalism on this site. You are seeing a decline in the fishing grounds, not necessarily a decline of the fishermans ability to fish. Its been happening for awhile.
-your friendly neighborhood shill-o-matic
I don't know, but if you think of it, you're going to need something like the total storage space of everyone else on the internet.
"First they came for the slanderers and i said nothing."
And that was over ten years ago.
Let me see if I understand this.
If I pick up my phone in Clito, Georgia and call my uncle, it will be recorded because the FBI has installed a splitter in the Bulloch County Rural Telephone Coop local office? And where, exactly, do they tap my 56K modem signal when I'm downloading gif's from the bulletin board that uncle Harvey has setup?
Nothing to see here, the Republicans did it.
Every time that it's liked, re-tweeted, emailed, hotlinked, or otherwise used elsewhere, It's more or less recorded as "At [TIME/DATE], mianne viewed phantomfive's cat video obtained at prior {TIME/DATE]."
That's not possible without deep packet inspection to know the context of the data transfer(s)
For example, if I upload a picture to my personal website using SFTP, there is no way to even know what the data is without inspecting and decrypting the packets. Even if you figured that out, once the file is on that server, I (or the server itself) might move or rename it, so any future retrievals with HTTP would pull a different filename. That's a trivial example, but it basically shows that matching "upload" with "download" isn't really possible.
Facebook, etc., would be just as much of a problem, as the system would have to track the upload and figure out what Facebook ended up calling the picture, since you can upload pictures with the same filename and have them show up as separate pictures.
And how much if you only record each youtube video once? . And log only meta information about vpn/ssl/https traffic. (source / dest / time...) And just ignore netflix, hulu, itunes, and the top 10,000 hollywood torrents...
Thank you for giving me great ideas on how to safely communicate with my terrorist buddies.
Kidding about the "my terrorist buddies", but the point is that you can't discount any data transfer as having the information that you need to stop/solve a crime.
So do they store all those 5GB Linux install iso's I download? It must be boring reading
Mod parent up.
The real transrcipt makes a lot more sense....
But much of that is streaming video from a few sources; that's easy to recognize and eliminate as containing no interesting data.
There are more things to consider than simply storage capacity. This system would have to be able to make decisions about the communications it receives exactly like a human would. It would have to know when something is a duplicate even when the MD5 doesn't match. It would have to understand natural language as well as an agent would and know how to sort and tag the data. It would have to do these things significantly faster than real time to account for peaks in traffic. It would also need to scale in computational power, bandwidth, and storage capacity at a rate at least fast as communications have. Hiding a system like that wouldn't be possible. You could find the existence of such a system just from the hard drive market alone. Consumer hard drive prices would be significantly higher because a large percentage of world wide manufacturing capacity would be consumed by this project.
No, re-read his comments. He only mentions that they are recording all voice traffic, not all data. He goes on to say that all digital communications is insecure, but not that they're actively recording all data traffic.
Voice comms is very low in bitrate, and it hasn't scaled up exponentially like general internet traffic, so I have no doubt that the technical capability to do what he says exists. Whether they are actually doing it is a separate question.
-->"no digital communication is secure", by which he means not that any communication is susceptible to government interception as it happens (although that is true), but far beyond that: all digital communications - meaning telephone calls, emails, online chats and the like - are automatically recorded and stored and accessible to the government after the fact. To describe that is to define what a ubiquitous, limitless Surveillance State is. --
what is that, if not claiming that your wow chat gets recorded by them?
world was created 5 seconds before this post as it is.
Someone correct me. It just doesn't seem feasible.
Speech recognition is pretty good these days, and text is highly compressible. If they discard the audio, It would be quite feasible to store a transcript of every call.
Verifying this is easy, just go to any Hard Disk factory and look for an endless stream of trucks to an unmarked facility. If you can find an infinite stream of HD's disappearing in a secrecy black hole, you got your proof.
Else... how the fuck are they storing it all? And it isn't just one secret facility. Local calls are... well... local. The entire setup is that a call never goes further then it needs to. So where is either the massive fiber to route all calls everywhere to a secret location only marked by an endless convoy of trucks OR all the local gigantic secret installation to record local calls.
Don't forget that even total surveillance societies didn't work and they usually didn't have free local land line calls. Just recording your average teenage girl would take more storage then there are molecules in the universe. A
And if it exists... well... what are they using it for? Where are the secret trial, the people disappearing? Oh there is Quatenomo but the point about that one is that for all its secrecy it is far from secret. If you have a secret government that knows everything about everyone, people should be disappearing left and right. And they are not.
The sad thing for the conspiracy nuts is that if there is a secret world government, it is a damned incompetent one. And the aliens ain't much better, either they crash or are reduced to abducting rednecks and cows. If I had super advanced technology capable of traveling the stars I would have better things to do with it then scaring hillbillies.
It seems that the Illuminati aren't that enlightened after all. Can you imagine being finally being granted access to this most secret society and then be told to spend all of eternity monitoring the calls of one suburban street worth of calls?
Phone call is about 64kbps(*4 one for each side... (or 128kbps for you to send/receive))
so a vastly smaller fraction of your calculation.
Why do moronic moderators keep modding up idiotic statements such as these from people who think absence of evidence is evidence of absence? Who ARE those guys, here "Charliemopps", that they think they should have known everything? Since when is THEIR cooperation and knowledge necessary? Even if they knew all that was going on everywhere in their OWN company (which not even the CEO does), that company is not connected to others, right?
This comment is STUPID STUPID STUPID.
Sorry, I've had enough after reading through them and now can't help it. I'm not all that mad at Mr "Charliemopps", everyone is entitled to say something very stupid at least once every hour, but that a number of random people who happen to be mods today find this garbage "informative"...
But cell phones typically use only 8Kbps "on air". This means that a factor 8 compression of "uncompressed" 64Kbps is feasible and that's probably what they achieve, maybe even more, if they wanted to record it all.
I was promised a flying car. Where is my flying car?
In telco land, they count each connection, even to an answering service, at least one minute. Actual talk time is much less than those 459 minutes. It's probably less than 2 hours. Any silence during those calls is not recorded, you probably have less than 60 minutes per month to record and that is per two telephones, since there's nothing to record if there aren't two telephones connected to each other.
I was promised a flying car. Where is my flying car?
Storage is not an issue if you only record a transcript of each phone call. Include a separate database of a low quality sample of parts where the voice to text transcript algorithm is unclear and higher quality samples of any phonecalls with preselected keywords or names or numbers mentioned/used.
Search and analysis is not a problem after the fact, you just identify people and keywords of interest, by which these are tagged and indexed.
I believe UK and USA hold such data indefinitely. And there have been several stories of USA building huge datacenters in deserts holding and analyzing this data. Also the algorithms and people doing this work is very old news.
And the goal is to record all the planet's conversations. Facebook and Google help map the people's connections and tag them to databases for any links.
The problems from using these for any FBI purpose after the fact comes from the fact that machine transcripted phone calls are hardly 100% accurate and won't hold evidence value in court, but certainly they should help know the topic of the conversation after the fact, which is what the FBI dude here is talking about.
If you are more interested, order the archives from: https://cryptome.org/ where I believe a lot of the news items related to this topic are archived.
captcha = echelon I shit you not. http://i.imgur.com/4QD6KOG.png
Nobody gives a fuck about your goddamned captcha.
The US has been recording everything for decades already. A few days after the KAL 007 plane was downed, the USA released transcripts of the voice comms between the fighter pilots. Nuf sed.
Excuse me, but please get off my Pennisetum Clandestinum, eh!
You mention deduplication like it doesn't mean much. I suspect the vast majority of data transfer is redundant. A surveillance entity would be more interested in one copy of your post than 100,000 copies of a youtube video with funny cats.
Broadly speaking, this is not wrong. The NSA are not really allowed to do this (they are supposed to be outward-facing, inward-protecting). Formally they have a reciprocal 'data-sharing' arrangement with the UK's GCHQ in Cheltenham: the practical upshot is that they both operate domestically, even though they shouldn't, via this loophole. The FBI may have been given some limited access at times, but this is not widespread (the NSA doesn't really like the FBI). The FBI could not use any of the surveillance in criminal investigations in any direct way: firstly it would be 'poisonous fruit', secondly it would reveal the existence of the programme. They have used it to direct some high-level interrogation and other operations however.
Not ALL internet traffic is recorded, but a significant proportion of it that passes through major POPs in the US is. The devices that do the monitoring are made by Narus (in all but name an executive division of the NSA), and perform DPI to determine what to record and what to discard. Some of them are capable of MITM but most are passively-spliced as active attacks would reveal their existence. All communications data (source, destination, time, context) is recorded. Streams are reconstructed wherever possible. I believe all unencrypted data captured is recorded, although dededuplicated (they'd have every YouTube video that's been watched, but not more than once). Encrypted data is selectively recorded (key exchanges for possible later use, depending on PFS status; encrypted streams of selected targets are also recorded).
As for encryption: they can crack 1024-bit RSA, but there is a long wait and this is only applied selectively and retrospectively. They cannot yet crack 2048-bit RSA/DH/DSA, or secp192r1 or above. They do not have cracks for AES, any of the AES finalists, any of the SHA-2 functions or SHA-3 finalists: in fact, in crypto, they are not really ahead of civilian academia anymore, and attacks like impossible differentials came completely out of the blue for them. The secret Suite A algorithms are weaker than Suite B and urgently need phasing out as they are insecure.
Yes, they really do have that much nearline storage. It's been joked to me that the United States Government is probably the biggest single customer of hard disks in the United States. What did you think they built that new datacenter for, pictures of cats? (Ironically yes, basically all the pictures of cats would indeed be on there. Snicker if you will, then realise this system was built on a what-if basis: what if terrorists were using them to communicate plans via steganography? Not that they'd likely find that out beforehand, but if they found out via HUMINT that this was happening they'd be able to look back and reconstruct things.) Think of it like a bigger version of Google Cache. It is deduplicated, but (unlike Google), it is very poorly indexed. Searching it is a complete nightmare and requires highly specialist software. It is mostly held for later data-mining for software like ThinThread, which connects networks of people in graphs and allows drilling-down of detailed analysis from there, and is much more usable than its predecessors. Facebook was probably a dream come true for them. I expect Graph Search interests them immensely.
As far as other communication goes: All text messages and voice calls that pass central points are recorded and retained, in compressed form (G.992.1?). Email is monitored where possible. Some chat is monitored too, but the FBI were furious that things like chats in popular games, and Mumble, were not (the NSA pointed out Blizzard log all the WoW chats and all they'd have to do is ask with a warrant and they could do things perfectly well by the book, but the FBI don't need no steeenking warrants and threw a hissy fit, which was unproductive). Some (many) communications are missed due to network failures. All communications data on phone calls and text messages are normally retained by phone companies, and this is provided in real-time. All satellite ph
Anyone who uses any electronic communication technique to do anything illegal is a fucking moron. This has been true since "Watson, I need you". Actually, it's been true since before that.
Actually, one could argue that anyone who does anything illegal is a fucking moron, but that's beside the point. This is about monitoring and surveillance. Some disgraced, ex-FBI guy claims everything is being monitored. Sure. Maybe it is, but who cares? The fact that the information has to be beamed or otherwise transported should have made that obvious from the start. It's as absurd as two people shouting at each other in the street being mad when someone else hears them. They were shouting. In. The. Street. Fact is, with the right equipment, E.T.'s from another planet could theoretically listen in on your phone conversations, since your phone (assuming its a radio-based phone) is emitting photons at the speed of light, and in basically all directions. A POTS phone likewise sends signals without adequate shielding and therefore also emits a certain amount of radio-noise.
Was the ex-FBI guy wearing a tinfoil hat when he said all this, by any chance? Also, how do we know he isn't just a disgruntled jerk badmouthing his ex-boss? If what he said were true, wouldn't that fact be... classified? If this guy is divulging classified information, and they interviewed him, would he still be walking around free? Unless of course, they wanted him to say this. I'm not worried about it, though. I don't say anything sensitive over a phone anyway, not that I ever have anything terribly sensitive to say. If I did, I wouldn't for reasons stated above. Duh.
which is not *that* much, especially if you can tap into existing databases like Google's without having to duplicate them
I don't think people seem to understand just how cheap storage is, the amount of money our government is capable of spending, or the amount of datacenter space they own.
"Hurr, where du u stor it?!?!?!??!?!?!?" is a stupid argument. It was a stupid argument when it meant nothing in terms of ISPs logging everything (re: roughly around 2000); it's a stupid argument when talking about a superpower.
I would guess that 99% (or more) of worldwide data is replicated already, so that compression is trivial. Think torrents, porn, spam, and static images. The real question is how much is encrypted and/or new data. The rise of https probably put a dent in collection efforts, as does every step towards encrypting traffic. That still won't stop the government from collecting and then decrypting the data later, say if they think its worth it. But it does make it more expensive. Everything on the internet is almost certainly recorded. But its a mountain of data to analyze. The political implications are huge. The real question, is who gets to search the database? Secrets are only powerful as long as they are still secrets.
Just do like the LHC does. filter things at multiple levels, and this is doable for telephones.
You don't really need *all* the data.
Data coming from public terminals might be important, data coming from common houses might be less important as people are afraid of being found out..
Or watch for call-loops. A circle of calls where A calls B, then B calls C and C calls A might be suspicious. bonus points for watched numbers.
Or a series of quick calls from someone, or a chain of quick calls...
filter out calls between families, or between companies... depends on what you're searching..
Or just keep all the data for a week, and if no one collects it, and the algorithms do not signal anything, then discard the old one to make space..
There are a lot of possible ways to analyze all our voice/text data..
If we talk about Internet traffic, then you could optimize thing a little, for example discard content from youtube, BBC, as they already log everything, keep 4chan... there's a lot more data, but profiling helps a lot.
It's not the same as "everything", but if you ask me, it's damn close... and doable....
just my 0.02$
"I was gratified to be able to answer promptly, and I did. I said I didn't know." -- Mark Twain
A billion dollars can buy you something on the exabyte scale these days (at HD prices around $0.04 / GB). The way the government throws around money...
It may be difficult for them to capture *all* data, but I don't doubt that they could capture the vast majority of relevant stuff.
Think about all the people saying terrorism-related keywords when playing Call of Duty or Battlefield.
Not that anyone cares about another AC post - but I know the guy personally. I have worked in the INFOSEC arena for over 30 years. He's telling the truth. It's all captured, then voice recognition is used to dump to a database. It's not analyzed on the fly, but there is a keyword capability (think "Echelon"). The main factor is that it's available for "after the fact investigation". George Orwell was an optimist!
of men, and I can find a way to hang him with them. The objections due to the idea they can't analyze it all mean nothing whatever - that would only be useful for you know, actually catching terrorists etc, which they've shown little competence for. No - the threat here is once you become a "person of interest" for whatever whim - they now have your whole past, and can surely find something on you as an excuse to harass/jail you. All those usually non-enforced discretionary laws now come into play, you know the ones they said we only use to bust those bums from out of town, those other people? Now, it's you.
Why guess when you can know? Measure!
holy shit, the whole internet is rife with stenographic messages disguised as spelling errors! Thanks for validating my schizophrenia!
the preceding comment is my own and in no way reflects the opinion of the Joint Chiefs of Staff
Except network-tracing ends at the nearest NAT, and controlling the true informationflow is an impossible and endless task (just ask the RIAA and MPAA mafia).
The US government can manage huge enough data for these assertions to be true for all practical purposes. They have purchased the largest datacenters in the world. At the scales we're talking about here, anything becomes possible.
In fact, a few years ago when I heard about the new datacenters in Utah, I told all my friends they were doing this then. Why spend such money if not to gain superior capability for all foreseeable future (10-20 years)?
Hints:
Audioconversations can be translated into text and indexed.
HTTPS can be intercepted.
Text can be compressed at 80-90% ratio.
Smart people working on this finds practical solutions to it, and no doubt, Google is a big part of the Big Data-picture for the US government.
Not that I know, but the claim doesn't sound 100% solid. Wasn't there news last month that the feds couldn't eavesdrop on iChat converstations (not that I believe them). So, people can still set up encrypted conversations and unless the private keys are not in fact public it wouldn't matter? Then again, Clemente is saying that the traffic is recorded and stored, not necessarily on-the-spot decipherable. Given enough computing power and incentive, I suppose encryption can be broken anyhow.
Anyway, don't have private converstations anywhere anymore I guess.
Even if we had the technology to record every bit that flows across the Internet (Petabytes and Petabytes of data)... and a way to catalog and index it... and the storage to save it for a very long time... there is the human factor to consider.
The more humans that are involved, the greater the inefficiency, the slower decisions are made, and less gets done. Information is compartmentalized, as information translates to political power, and influence. The kind of organization required to operate a system of this magnitude... only exists in movies, and in the minds of the conspiracy theorist, who has little experience dealing with large organizations.
Look at the Boston Marathon debacle. The inter-agency squabbles were in full play. The only reason we caught the guys at all was due to the vigilance of a few ordinary folks. The billions doled out by DHS to the states and cities produced a lot of fancy paramilitary equipment being driven around, none of it was all that useful. After the event far more resources are being expended spinning the wheel of blame, as everybody tries to cover their asses.
Murphy was an optimist
I'm waiting for Bruce Schneier to weigh in on this. If he confirms it, it must be true. He's worked with the spooks before. On the other hand, if he denies it it must be true. He's worked with the spooks before.
Chelloveck
I give up on debugging. From now on, SIGSEGV is a feature.
Jesus, this was barely a year ago:
The NSA is building the world's largest data center in Utah. They are expected to be able to store a lot of data there, some of it for up to 100 years.
Also they OBVIOUSLY don't store everything that flows across the Intertubes, that would be moronic. They use standard DPI techniques, and probably a bunch of classified fancy DPI techniques, to pick out the 0.1 to 1% most interesting traffic, and store that. Presumably they store records of who is communicating with whom at any given time, but they don't store entire copies of those pirated .mp3's you're downloading, because why bother? But presumably any individual IPs, e-mail addresses, domains, etc. that they have identified as "interesting" or "possible threats to national security" have a lot more detail stored about them. If you visit known bad-guy sites or fetch content that their heuristics suspect is more deserving of scrutiny for whatever reason, then expect all of that traffic to be stored and analyzed by automated algorithms.
This is just common sense, especially since after 9/11 they knew they could get away with it.
Don't you remember Bill Binney?
Don't you remember Thomas Drake?
And if 15-minute videos are too boring for you, watch this hilarious (and chilling) rap news video which is only 6 minutes.
This is VERY OLD news, why did Slashdot take over 10 years to report this? This actually came to light when the first FBI tap in AT&T's dark room came to light ...
Comment removed based on user account deletion
a) FBI has a time-machine
b) P == NP
Kidding about the "my terrorist buddies", but the point is that you can't discount any data transfer as having the information that you need to stop/solve a crime.
If you downloaded it from netflix, hulu, itunes, or amazon... how do you plan to embed your communications? You don't control the file, you can't modify it.
Unless you think netflix is a terrorist organization distributing movies with embedded communications. But it wouldn't be too hard for them to rule that out.
Meanwhile, as long as you have the hashes for the files, you don't even need to store the file. You can validate netflix sent party X a copy of Joe Dirt... so why bother permantly saving a copy?
This sounds like the answer to a Google Job Interview question.
Whether or not they are ACTUALLY storing everything is rather moot, since you can't tell when they are and when they're not. It's safest to assume everything is recorded, and that way you'll never be caught by surprise (nobody expects the FBI inquisition!). It's also prudent to assume that any encryption you are using will be cracked within your lifetime. Not that you shouldn't use it mind you, at the very least it might force them to earn their pay.
But why don't we all see if we can get Congress to raise their budget for hard drives by transmitting terabytes of hard to compress random noise to each other as if it were something important? We can prove they're trying to store it if all of a sudden the budget gets raised in the face of massive mounting debt and the sequester...
All of my encrypted garbage gets to be in their database.
I'm glad this is what my tax dollars pay for! Creating digital garbage.
captcha = echelon I shit you not. http://i.imgur.com/4QD6KOG.png
Nobody gives a fuck about your goddamned captcha.
That's what I mean. You're all dead inside. You tell your stupid jokes from 15 years ago and seem to be stuck there when it comes to the rest of reality.
Anybody read about the gigantic data center being built in Utah? It is being used to store all electronic communications between embassies and foreign governments, most of them encrypted. The plan is that decryption will occur in a few years and they can decrypt the files and gain insight into the thinking of foreign governments.
That makes a lot more sense (because it will never work as planned. same old same old). By the time we decrypt the files, there will be new governments in place, both here and there. But for now, no one has enough computing power to both store and analyze everything communicated digitally.
If you downloaded it from netflix, hulu, itunes, or amazon... how do you plan to embed your communications?
My sleeper agent who was specifically trained to get a job at one of those places (although Facebook would be the best) will do the embedding. If I was organizing any sort of wide-ranging criminal activity like terrorism, getting somebody inside one of those big companies would be one of my bigges priorities, as it would allow covert communications with almost no chance of detection.
Meanwhile, as long as you have the hashes for the files, you don't even need to store the file. You can validate netflix sent party X a copy of Joe Dirt... so why bother permantly saving a copy?
Doesn't Netflix vary the quality based on the bandwidth? If so, there is no fixed hash. In addition, you now have to deal with some sort of live database that sums up the packets from Netflix to the destination, but only those packets that are the movie (no control packets), and keeps the running checksum/hash. Without deep packet inspection, this isn't possible.
My sleeper agent who was specifically trained to get a job at one of those places (although Facebook would be the best) will do the embedding.
If I was organizing any sort of wide-ranging criminal activity like terrorism, getting somebody inside one of those big companies would be one of my bigges priorities, as it would allow covert communications with almost no chance of detection.
How many employees do you think can just go in and modify the live content files at netflix, propogate the changes out all the ISPs where the data is locally hosted, etc.
And getting a sleeper agent who can do that is going to be your top priority? Good god what on earth for? What on earth are you going to distribute via sleeper modified netflix movies that wouldn't be just as easy to distribute any of a hundred other much simpler ways? You'd have to activate the sleeper so theirs risk of being tapped at that point if they are watching you. And you'd have to have some way of messaging the troops to download a particular movie to get the embedded instructions.
What does that really get you that some guy posting cat videos on youtube doesn't?
So does using SSL from a wifi hotspot.
Doesn't Netflix vary the quality based on the bandwidth? If so, there is no fixed hash. In addition, you now have to deal with some sort of live database that sums up the packets from Netflix to the destination, but only those packets that are the movie (no control packets), and keeps the running checksum/hash. Without deep packet inspection, this isn't possible.
Fair enough, they could have the interception/filter/logging mechanism right in front of the netflix servers. Makes things simpler. And netflix is a big enough data flow they could eliminate a lot of traffic by handling it directly at the source.
..that they're recording or analyzing my wife's conversations over video phone in ASL. If they want to record when I order a pizza over the phone, they're welcome to it.
Really, so the Government needs to store 4,267,392 copies of that stupid cat video that people watched on youtube?
You are SEVERELY underestimating the ability to dedupe a constant stream of network traffic at the backbone level.
You'd have to activate the sleeper so theirs risk of being tapped at that point if they are watching you. And you'd have to have some way of messaging the troops to download a particular movie to get the embedded instructions.
What does that really get you that some guy posting cat videos on youtube doesn't?
That's why Facebook would be the best, but Amazon or eBay would also pretty good.
For Facebook, you could add steganography to photos of any account that allows "open" followers (i.e., not an individual). For Amazon or eBay, you could do the same to the user-provided item photos. The fact that the photo isn't the same as the original isn't suspicious as these companies already alter the photos in various ways.
The reason for a system like this instead of posting to YouTube, 4chan, etc., is that the poster could be under suspicion, and thus everything they do would be watched. Posting from inside is harder to track by outside authorities. In addition, you could alter enough photos with the same message that no two recipients would need to look at the same Facebook page, Amazon item, or eBay auction. That means that even if the recipients are under suspicion, their actions couldn't be correlated.
It also allows easy innocent explanations, like somebody looking for messages in photos of concert T-shirts on eBay could actually be buying and wearing a few of the shirts, and that cover could go back several years. A extra advantage to this would be that the completely innocent sender of the T-shirt might come under suspicion if the recipient was deemed suspicious. This isn't the sort of planning you would do for something one-off, but rather for a concerted, long-term effort. This is the kind of cover that agents built over many years during the Cold War.
Oh well, 9/11 was the Reichstagbrand of the Fascist States of America.
The only thing I wonder about is that there aren't half a dozen uprisings and civil war going on, with a dozen domestic terrorist attacks every day -- with the express goal of liberating you from your fascist regime. Instead all you've got is some lone madmen and a few FBI fabricated "foreign" terrorists. Must be working well, then.
"The more prohibitions there are, The poorer the people will be" -- Lao Tse
you have the internet!!!!!
of course the question is, why do you need to reply to me on slashdot?