Slashdot Mirror
US Government Upgrades RAM
Deep Throat writes "Techworld has the scoop on a new super-sized RAM disk that the US government has just bought for a few million dollars in order to speed up searching through huge databases. It's 2.5TB! The VP of the company that made it says it is for Washington DC and searching databases but won't say who. Techworld explains why it reckons it's the Department of Homeland Security searching in the NSA and Pentagon databases for terrorists. And apparently the government is 'very happy' with the purchase and thinking about getting more."
Before many users start discussing the privacy laws and what not, it should be noted that the data being stored is probably not new. It's the medium on which it is stored on, which is.
Even without this, the old database could have been searched for some terrorists. Nothing has really changed.
While a good point, perhaps, I'm sure the government has no interest in even considering that unless they have the necessary government classification clearances, which I'm guessing would be pretty high given the assumed use of a database of this magnitude.
-N
I've nothing to say here...
More likely, the NSA's already got a dozen of these things and loves them...
Yes Francis, the world has gone crazy.
Perhaps we should begin to consider the implications of this step in analytical ability. Every techie knows that the Government (in cooperation with major Corporations) has had the methods in place to track individuals... RFID tags, GPS locating (in automobiles as well as hand held units. You can tell your position, but the position is also transmitted back... Five Star anyone?), Cell Phone triangulating, and thanks to the Patriot Act, the Government now has legal access to the records of pretty much any transaction we make with bookstores, libraries, etc (and probably more places as well... and this isn't even taking into account information that they might be recieving in ways that we do not know of).
The thing that (we shall assume) they didn't have before was the ability to instantaneously cross check this information. (I assume this because... well... why would they have bought the drive otherwise?) Now that they can check such information so quickly, will we be brought into an era of "Total Information Awareness" as the government spoke of not too long ago? Does this smell the same as the Thought Police to anyone else?
To me, this presents at least the intent by the government to achieve total information awareness, if it doesn't actually achieve it. And the intent is bad enough. Perhaps they're not reading our minds, but the ability to monitor our actions in such vast varities of levels comes pretty close to doing so.
-Vendal Thornheart
Can I get some sources to document this?
Actually no, Google crawls a vast amount of information but a very small percentage of that information is useful (specifically the links and keywords). Google also remebers common queries and plays other tricks to speed up searches. When your looking at a 2.5 TB database it is very different. First of all history doesn't matter as much since it is unlikly that they will be running multiple searches on the same thing. Second the 2.5TB is all meta data, so its all relivant, so it all must be searched. Third a query is likly to be much more complex then a normal google query. Google has one index that can be clearly defined in an alphabetic way. The govenment has many data bases that are indexed multiple different ways. This makes searching for connections between databases very difficult. In short Google is highly optimised for a specific type of search and probably will not work in the much fuzzier realm of inteligence.
True, but as I said... (Possibly in another post) Google has some very good people working for them. Like, a sizable number of the major contributions to graph theory research over the last ten years major. I'm betting the USG could also deliver a (deliberately fuzzy) list of requirements to them and get back something that'd do what they wanted.
Putting the right indexes on a table with 100 million records to satisfy a very small set of well-documented expected queries is not exactly rocket science. And I think you underestimate what Google does.
This is probably just a bunch of govs sitting around going, well, we got this new budget, how are we going to spend it. And one guy said, I bet it would be really cool to have a 2.5 TB RAM
Anonymous Cowards - Oh God, How I hate you
Echelon
Thats a freaking BLOG!
How can you possibly believe that? True disk accesses are slow, but it doesnt mean that they have to be completely avoided to still get good response time.
Caching everything in RAM definitely helps, but is not feasable. There are many other concepts out there that Google must use as well (dedicated RAID disk subsystems, metafiles, heuristic-based searches, etc...)