Slashdot Mirror
Can Commercial Storage Services Handle the NSA's Metadata?
itwbennett writes "In a review of NSA surveillance last month, President Obama called for a new approach on telephony metadata that will 'establish a mechanism that preserves the capabilities we need without the government holding this bulk metadata.' Obama said that a third party holding all the data in a single, consolidated database would be essentially doing what is a government function, and may not increase public confidence that its privacy is being protected. Now, an RFI (request for information) has been posted to get information on U.S. industry's commercially available capabilities, so that the government can investigate alternative approaches."
And what if some commercial storage vendor can't or won't handle the NSA's metadata archiving requirements?
The world's burning. Moped Jesus spotted on I50. Details at 11.
It's the only was to be sure.
This is less of a technology problem than a policy question. The technology exists to build secure databases and make it accessible to only one remote client. The real controversy is over collecting the data, and who holds it. Private companies don't want to do it. Many are against the NSA, and by extension the Federal government doing it. If only there was somewhere in the middle, between the Federal government and private industry...
much of left-wing thought is a kind of playing with fire by people who don't even know that fire is hot - George Orwell
Given enough money.
Once the USA government asks for bids on this, you will get many companies wanting a share on this juicy contract. This is supposed to be with the intention of increasing security, but just wait a couple of years and stories will start to pop up as to how corners have been cut to turn a few extra dollars with the result that this data becomes available to all sorts.
Instead of a government theoretically beholden to the Constitution theoretically being held responsible for their actions, we get a corporation practically beholden to nobody but its shareholders selling the information to practically all comers (LexisNexis, anyone?)
In theory, this is shit. In practice, it's worse.
It's a bluff. A feint. A thinly veiled threat. It's not intended to actually come to pass. One of the things Obama proposed is to move the keys to the friggin kingdom from government controlled servers to nebulous "third parties". And in the very same damn speech he pointed out how this would be a ludicrously bad idea.
(Well, I mean, he also suggested that the telcom companies who move this data keep it until the NSA asks for it. That or third parties. I don't mean to harp on a stray comment or anything.)
But let me spell out the subtext here for anyone that can't read between the lines: If you try and keep the government from storing this data, we'll just go find someone else to hold it. And my, my, my, doesn't that sound just simply horrible? Be a REAL SHAME if someone were to try and enforce that 4th amendment 'round here.
Also, fuck beta. I have no way to tell if someone responded to me other than looking at that specific thread.
Ask a USENET service provider like giganews, they know the drill.
America should go for Mongo DB...its web scale. And in addition its "high performance" and supports sharding.
Tat Tvam Asi
OK, so they want to store everything passing across the lines that they deem suspicious, promise us that no one will look at it with a warrant, then if you're ever suspected of something they can go back and find all your communications over the past X years. And, since the feds don't want the blame for holding onto this information (and looking as Big Brother-ish as they are), they want private industry to pony up the disk space? I'd almost trust the NSA more to house this info since they'll only snoop in on my conversations when I post/say a flagged word/phrase. Wheraeas I KNOW private companies will as soon as they figure out how they can commoditize it.
It's Orwellian enough seeing Google spam me with ads based on my email conversations.
I swear to God...I swear to God! That is NOT how you treat your human!
They probably can, given enough money, but 'the capabilities they need' are actually quite modest. The metadata program has no legitimate utility, so just write me a check for half a billion, and I'll build a machine that sits idle and is not connected to the internet, let alone accessible by the NSA. I've solved your problem with equal efficacy and far reduced cost.
This is my signature. There are many like it, but this one is mine.
Can they? Sure. It's not as though the private sector can't store data, if provided with the right incentives. Heck, AT&T is providing the DEA with access to nearly three decades of call records, plus consulting expertise, right now!
Trouble is, that was never the fucking point. Do people want the NSA collecting a giant database about them? No. Does it make the slightest difference if the giant database is nominally Verizon's giant database, that just so happens to respond to all queries from the NSA? Aside from the greater likelihood that the database will be used for marketing and surveillance, not a bit. The ostensible '3rd party' won't remain at arm's length for long. Why would they? An entire organization with a single customer, dedicated to shovelling data toward them on command? Instant capture. The only time the 3rd party will be 'independent' is if somebody asks the NSA what that 3rd party is up to, in which case they'll oh-so-innocently-have-no-idea-what-that-independent-entity-does. For all other purposes, they'll be joined at the hip.
Another way to illegally store people's information. I guess being the president he feels he doesn't have to follow the law or is subject to being held criminally responsible.
The problem isn't where the collected data is stored. The problem is that it is being collected. There is no reason that the bulk metadata of every phone call made in the US is stored for years or indefinitely. There is no need for this RAW data to be shared with other countries. So where it is kept makes no difference. This data shouldnt be kept at all, and from every independent analysis of the program it has had NO impact on fighting terrorism. So it is a colossal breach of the constitution and a massive waste of money and resources.
...well enough to be leaked.
Google and Facebook have the talent that, with a several billion dollar (per year) federal contract they could probably be incentivized to put together a team and plan to make it happen. They have the infrastructure already. They just need enough coin to make it sufficiently attractive to work on the problem.
But the real problem here is uglier than this. The NSA program is the price we pay for living in a globalized, "open society" that prides itself on not doing things like "profiling" and that is unwilling to establish sufficient border and immigration security to reduce the risk of hostile foreigners stepping foot on our soil.
If you try and keep the government from storing this data, we'll just go find someone else to hold it. And my, my, my, doesn't that sound just simply horrible?
Back in the 90s, I dated a lawyer. She said that with someone's SSN, she could find out everything about them.
Today in '14, not only do we have people voluntarily broadcasting their personal details, but the financial, medical, and retail industries has an obscene amount of data on all of us. And it's aggregated already by the credit bureaus and companies like ChoicePoint as well as search engines like Google who can do it on the fly. And the Medical Information Bureau has your health history. The phones companies have your calling history.
And let's not forget the intrusive information gathering by our governments.
Anyone who claims that your information is confidential is only speaking about some schmoe off the street. If you pay for it or send a real scary letter with law enforcement letterhead, all bets are off.
So, my point? The NSA's plan is redundant. It's be much cheaper just to force the above companies to do their bidding.
And if Google (Brin) or whoever don't like it - oops! Some N. Korean or Russian submarine mistook his yacht for a terrorist launching point. Our bad!
The problem isn't who holds and maintains the data. The problem is that some entity is collecting in advance protected data.
The real method should be if a person has a signed court order granting an investigation against them then you can begin monitoring.
Innocent until proven guilty; Unmonitored until signed court order has been issued.
Please update laws accordingly thank you.
Now you have to worry about the security of the third party. I give my credit card to a third party (say Target), Target follows the rules on what can be persisted, and yet you still have a data breach.
The meta-data information provided by the President is a fucking cover story for hiding their spy games program. It's already been exposed that they are doing much more than saving meta-data; they're collecting word for word, every communication domestically and foreign, saving the content of our communications.
Lets focus on the meta-data for a minute thing: according to Bill Binney, previous NSA director on technology that helped design the system, anybody can store meta-data and equipment that fits inside a 20 by 12 foot room. FOR ALL COMMUNICATIONS, WORLD WIDE. So of course Verizon, AT&T, and these others douches can store this information. In a room probably the size of 5 by 5, because they'll be storing it themselves ; and providers are already storing this information anyway, which has been available for law enforcement use for some time. The Bluffdale data center in Utah is big enough to store 100 years of content data though, .. which means they're using it to store actual profiles and content of people, not just meta-data. Details @ http://www.pbs.org/newshour/bb... "NSA Collects ‘Word for Word’ Every Domestic Communication, Says Former Analyst"
On top of that, they have a massive satellite and radar system with a variety of capabilities, which is being used to target Americans during continuous black operations. Mind reading capability, tracking from space, watching our movements wherever we are. look at the details @ http://www.oregonstatehospital...
Fixed that for ya.
The world is globaliszed, don'tchaknow? I'll bet some Chinese firm would have *no* trouble offering to host the outsourcing of the data storage....
mark "on Chinese-made chips...."
Sure. Let's not shut down the horrible program that a ton of people oppose and instead hand the data over to a company to manage and keep secure. What's the worst that can happen?
Off the top of my head:
1 - Hackings. No database is secure. If anyone was to store the data securely (putting aside for the moment the question of whether they should have the data in the first place), I'd trust the NSA to do it over some random company. At the very least, this reduces the potential attack vectors.
2 - Profits. The company controls this data and realizes that they could make a ton of money off of it. Their federal contract might forbid it, but that's easily handled with a few lobbyists and sneaky riders on must-pass bills. Now, they can sell information to third parties legally. Maybe it's aggregate data/not personally identifiable (at least, at first to reduce any opposition) and maybe not. Either way, this information is now leaking out.
The answer to all of this, of course, is the answer to the question "Why does the NSA need to store metadata on EVERYONE?" They don't. However, they have fallen victim to a combination of lust for power and a "information gathering" fallacy. (Collecting some information proves useful against terrorists therefore collecting ALL THE DATA will prevent all the attacks. Except that they've just increased their signal to noise ratio to the point that they can't spot the tiny number of terrorist signals within all of the random noise.) If they scaled the program back to only collect metadata on a very limited number of individuals (proven to a judge enough to issue a warrant and with checks and balances to prevent abuse), they would have a higher signal to noise ratio and might actually catch more terrorists than from a random sweep.
My sci-fi novel, Ghost Thief, is now available from Amazon.com.
I would assume that the methods used to collect this data are CLASSIFIED. Why else are they trying to get their hands on Snowden for leaking some of it?
IF you have classified information to store, you DON'T put it on third party systems unless they are under the necessary controls required to handle classified data. So, putting this data on contracted storage is NOT going to involve calling Amazon AWS for an account and just copy it up and pay the bill. So in reality you'd just be contracting somebody to build and run a storage solution for you.
Now *could* the government go out and *contract* with somebody to store their data someplace? Sure, it might even make sense to push it off to a number of contractors, but you NEVER, (and I mean NEVER) put classified data into public view (i.e. on systems you don't directly control), even encrypted, unless you have no choice. If you do, you are being STUPID. The more sensitive the information, the more this is true.
Assuming you don't use a one-time pad cypher, encryption doesn't mean that the adversary cannot read it only that they will have to break your encryption to see it. Brute forcing a key is *always* possible, the question is really "How Long" will it be, on average, before they will be able to view it, because they will eventually be able to.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
The answer is "No."
Seriously, all your data is perfectly safe. I have worked with GIS for 14 years. and I can tell your conclusively that absolutely no one reads metadata. :)
Let me list the ways. Oh, wait, there aren't enough bytes in the universe to expand this! 1. We only need to hack one site to get EVERYTHING! 2. See number 1.
http://devnull-as-a-service.com/
I vote that these guys should get the contract.
In the beginning was the Denelcor HEP. After Chapter 7, the chief scientist spent time in "Maryland" designing the successor. That became the Tera MTA. Tera became Cray, Inc, and the MTA system became the XMT. XMT was never a volume seller, but Cray would build one if you had the cash, and three-letter agencies did.
Now Yarcdata is a Cray subsidiary marketing it as a "graph appliance." See yarcdata.com.
Yeah, Mod parent up. Beta sucks!
Fire timothy "timmyboy" while you're at it.
the entire Internet will have unfettered access to the data, without actually being able to access said data, thanks to the perpetual irreparable nature of the system's design. ... just visit http://404.nsa.gov
I think the job should go to the same team that built healthcare.gov
That was easy. The commercial storage vendors do a very good job of pairing their spindles and flash with technologies from Oracle and various big-data vendors to make things work quite nicely. No issues at all. I've seen systems that can ingest many TB/hr. without problem.
...at the request of the State is working as an Agent of the State. As an Agent of the State, it is required to meet the exact same 4th Amendment requirements as the State itself. This whole argument is ridiculous. President Obama should be laughed out of office for seriously considering this proposal. Constitutional Law professor, indeed!
Does anyone really think that this will happen?
Come on, they've (Big Govt) spent billions (or more) on data storage facilities and you really think they'll just close those down and let some 3rd party store data?
I have an escalator to the moon for sale..
What the CRAP is going on with these comments formatting? Beta used to mean something was being improved but not ready for general use. Now it means crapping on users and destroying not only the super simple ease of use of a comments driven web site, but crapping on the users themselves in the process. Will someone show me how to change my homepage from slashdot to anything else!
and make the NSA's data collection program seem "cool", as if it's a cool technical consideration on how much storage it will take to store everyone's private conversation.
Beta is not the problem. While you turds complain about beta, these Psy-ops make their way to the front page of slashdot.
Have gnu, will travel.
Obama felt the need to say something. So he gave a speech in which he called for change. Problem solved.