Slashdot Mirror
GAO Studies U.S. Government Data Mining
securitas writes "Total Information Awareness is alive and thriving. eWEEK's Caron Carlson reports on a new General Accounting Office study that says TIA-style data mining programs are rampant in federal agencies with 199 projects at 52 of 128 agencies. The Defense Intelligence Agency/DoD is the single largest user of these data mining projects (eg. Verity K2 Enterprise). The story was first reported by Reuters' Andy Sullivan (ZDNet UK mirror) and the NYT's Robert Pear, who wrote that at least 122 projects used personally identifying information like names, e-mail addresses, Social Security and driver's license numbers. The 'actual numbers are likely to be much higher' because the report excludes classified projects. Wired News' Kim Zetter writes that, in addition to government databases, federal agencies mine private databases of credit rating agencies, bank account numbers, student loan applications, etc. This week the Center for Democracy and Technology (CDT) released a report with privacy guidelines for data mining technology (PDF) development and use. Guidelines include data anonymization, government data access authorization and audit trails. Cynthia (Cindy) Webb's 'Total Information Dilemma' at the Washington Post is an excellent survey of media coverage of TIA, MATRIX and the GAO report 'Data Mining: Federal Efforts Cover a Wide Range of Uses' (mirror, both in PDF format). More at GCN, GovExec and the Guardian/AP."
In Germany (no, data does not search you) it is forbidden to gather more information than necessary. We call that principle "Datensparsamkeit" (data frugality).
And it is not allowed to give information away or even sell them (that is only allowed for public available information)
Forget TFA, the summary is complex enough as it it.
A summary of the summary is though:
Wired News' Kim Zetter writes that, in addition to government databases, federal agencies mine private databases of credit rating agencies, bank account numbers, student loan applications, etc. This week the Center for Democracy and Technology (CDT) released a report with privacy guidelines for data mining technology (PDF) development and use.
3: Our economy is going to shit and we're going farther and farther into that hole.
GDP is growing faster. Unemployment is going down. Manufacturing is strong. The stock markets are moving up. Now by what measure does this translate to "shit"?
The facts are so clear the US economy is back on track again, why do these idiots keep sticking to their view from 3 years ago? This meme needs to die. It's not true, and it's certainly not insightful.
Slashdot Moderation: From positive to terrible in 2 "insightful" posts.
Absolutely true. I worked on a huge commercial/government mining and data integration project in the mid-90s (this isn't exactly something Bush thought of) and the state of the government databases were attrocious. Extremely dirty data, and of the over a thousand databases that were integrated, half the government ones looked like they had been maintained in a spreadsheet or flat file rather than any kind of real DBMS. A lot of the data was so out of spec that we discarded 10% of the records during the integration because it couldn't be unambiguously deciphered. A real data management nightmare; it is a wonder they can find anything at all.
By comparison, commercial databases are immaculate. The government needs to develop some type of professional data management corps. Private industry would think you were mad if you suggested managing data the way the government often does. Lots of one-off hacks by people who know nothing about basic principles of creating a database. This isn't just at the Federal level, the State databases were just as bad.
Back when I was doing work for the government, the open secret was that the quality of the data in the databases was so low that almost nobody used it because it couldn't be trusted. People were paid to maintain the systems, but very little utility was ever realized.