Slashdot Mirror
US Government Upgrades RAM
Deep Throat writes "Techworld has the scoop on a new super-sized RAM disk that the US government has just bought for a few million dollars in order to speed up searching through huge databases. It's 2.5TB! The VP of the company that made it says it is for Washington DC and searching databases but won't say who. Techworld explains why it reckons it's the Department of Homeland Security searching in the NSA and Pentagon databases for terrorists. And apparently the government is 'very happy' with the purchase and thinking about getting more."
Before many users start discussing the privacy laws and what not, it should be noted that the data being stored is probably not new. It's the medium on which it is stored on, which is.
Even without this, the old database could have been searched for some terrorists. Nothing has really changed.
In and of itself, $4.7 million is not that much money. It's when the government spends $4.7 million of taxpayer money frivilously on this and 10,000 other similar projects that it adds up.
While a good point, perhaps, I'm sure the government has no interest in even considering that unless they have the necessary government classification clearances, which I'm guessing would be pretty high given the assumed use of a database of this magnitude.
-N
I've nothing to say here...
More likely, the NSA's already got a dozen of these things and loves them...
Yes Francis, the world has gone crazy.
You have no idea what it's for. The list of known terrorists and their acquaitances is relatively short, I cant imagine more than a few gigabytes being needed.
Perhaps it's to store tax returns so the government can mail you your refund check faster. (Job required, sorry).
Maybe INS (or USCIS or whatever they're called) want to track the tidal wave of benifits being handed to Mexican illegals.
I'm a little tired of all this Big Brother speculation. Get over it.
I don't need no instructions to know how to rock!!!!
Sure, it could be used for good. But you know and I know that it will eventually be used for evil.
Sorry, but the best I can imagine is a RAID of those things.
The Tao of math: The numbers you can count are not the real numbers.
You can put a lot of personal data in 8KB.
:)
You assume the government is efficent
Perhaps we should begin to consider the implications of this step in analytical ability. Every techie knows that the Government (in cooperation with major Corporations) has had the methods in place to track individuals... RFID tags, GPS locating (in automobiles as well as hand held units. You can tell your position, but the position is also transmitted back... Five Star anyone?), Cell Phone triangulating, and thanks to the Patriot Act, the Government now has legal access to the records of pretty much any transaction we make with bookstores, libraries, etc (and probably more places as well... and this isn't even taking into account information that they might be recieving in ways that we do not know of).
The thing that (we shall assume) they didn't have before was the ability to instantaneously cross check this information. (I assume this because... well... why would they have bought the drive otherwise?) Now that they can check such information so quickly, will we be brought into an era of "Total Information Awareness" as the government spoke of not too long ago? Does this smell the same as the Thought Police to anyone else?
To me, this presents at least the intent by the government to achieve total information awareness, if it doesn't actually achieve it. And the intent is bad enough. Perhaps they're not reading our minds, but the ability to monitor our actions in such vast varities of levels comes pretty close to doing so.
-Vendal Thornheart
Can I get some sources to document this?
Actually no, Google crawls a vast amount of information but a very small percentage of that information is useful (specifically the links and keywords). Google also remebers common queries and plays other tricks to speed up searches. When your looking at a 2.5 TB database it is very different. First of all history doesn't matter as much since it is unlikly that they will be running multiple searches on the same thing. Second the 2.5TB is all meta data, so its all relivant, so it all must be searched. Third a query is likly to be much more complex then a normal google query. Google has one index that can be clearly defined in an alphabetic way. The govenment has many data bases that are indexed multiple different ways. This makes searching for connections between databases very difficult. In short Google is highly optimised for a specific type of search and probably will not work in the much fuzzier realm of inteligence.
Customs and Border protection (specifically, the old "Immigration" branch) could benifit from distributed computing for their database. They scan a car licsence plate and run a search while they ask the usual questions- where have you been, where are you going, got any meat? When the search completes they look at it and send them on their way. Quick searching could allow the CBP agent to notice that the car was stolen and that the drug dealer always carries a gun, so that they could wave them through and avoid conflict.
Even those who arrange and design shrubberies are under considerable economic stress at this period in history.
True, but as I said... (Possibly in another post) Google has some very good people working for them. Like, a sizable number of the major contributions to graph theory research over the last ten years major. I'm betting the USG could also deliver a (deliberately fuzzy) list of requirements to them and get back something that'd do what they wanted.
None credible. It's a conspiracy theory. It has been denied repeatedly by the United States government. That does not mean it is not true, no, but on the same token, NASA denies that they didn't fake the moon landings.
:P
I pay scarce attention to this guy's little fantasy, frankly, and walk away from people who start spouting this kind of stuff to me in real life. No "bye," no "you're a conspiracy theorist"; best to just walk away from the nut. Trust me.
Correction: you can put a lot of textual data in 8KB. If you start putting other things like fingerprint scans, voiceprints, DNA profiles, mug shots, and other things that aren't character-based, 8KB immediately becomes ridiculously tiny.
In the end they will lay their freedom at our feet and say to us, Make us your slaves, but feed us. - Fyodor Dostoyevsky
Putting the right indexes on a table with 100 million records to satisfy a very small set of well-documented expected queries is not exactly rocket science. And I think you underestimate what Google does.
This is probably just a bunch of govs sitting around going, well, we got this new budget, how are we going to spend it. And one guy said, I bet it would be really cool to have a 2.5 TB RAM
Anonymous Cowards - Oh God, How I hate you
Echelon
Look at Chile for a past example. Venezuela today too, although hard to say for certain.
That is why Google has multiple copies of the entire web in memory.
Interesting link, though I wonder if everything it claims is true. Specifically I'm referring to the business about every page Google has indexed necessarily being in memory simultaneously. Possible, but I'd have to hear it from a Google programmer familiar with the area to start to believe it.
And even if it were true, the statement that Google has "multiple copies of the entire web in memory" is certainly false because there are still many webpages that are not indexed by Google.
Thats a freaking BLOG!
How can you possibly believe that? True disk accesses are slow, but it doesnt mean that they have to be completely avoided to still get good response time.
Caching everything in RAM definitely helps, but is not feasable. There are many other concepts out there that Google must use as well (dedicated RAID disk subsystems, metafiles, heuristic-based searches, etc...)
Fingerprints can be (and are) encoded as a couple of vectors, voiceprints are similar. DNA profiles only need to record the magnitude of around a dozen markers to accurate to the person, mugshots are the only thing on your list which would require a lot of storage, and you don't need those for the searching part of the DB, just have them stored in a blob with a key from the main record pointing to it. Don't bring up facial recognition, it doesn't work, false positives are through the roof and false negatives are too high to trust.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.