Slashdot Mirror
GAO Studies U.S. Government Data Mining
securitas writes "Total Information Awareness is alive and thriving. eWEEK's Caron Carlson reports on a new General Accounting Office study that says TIA-style data mining programs are rampant in federal agencies with 199 projects at 52 of 128 agencies. The Defense Intelligence Agency/DoD is the single largest user of these data mining projects (eg. Verity K2 Enterprise). The story was first reported by Reuters' Andy Sullivan (ZDNet UK mirror) and the NYT's Robert Pear, who wrote that at least 122 projects used personally identifying information like names, e-mail addresses, Social Security and driver's license numbers. The 'actual numbers are likely to be much higher' because the report excludes classified projects. Wired News' Kim Zetter writes that, in addition to government databases, federal agencies mine private databases of credit rating agencies, bank account numbers, student loan applications, etc. This week the Center for Democracy and Technology (CDT) released a report with privacy guidelines for data mining technology (PDF) development and use. Guidelines include data anonymization, government data access authorization and audit trails. Cynthia (Cindy) Webb's 'Total Information Dilemma' at the Washington Post is an excellent survey of media coverage of TIA, MATRIX and the GAO report 'Data Mining: Federal Efforts Cover a Wide Range of Uses' (mirror, both in PDF format). More at GCN, GovExec and the Guardian/AP."
Just tell me which link to click so I can RTFA.
Post: Sigged, for your pleasure.
I live in Canada. Where only coporations do that sorta thing. NO Radioshak employee #1293 you do not need my address name and first born to sell me this 2$ gizmo
A psychopath can't tell the difference between right and wrong. A sociopath knows the difference - he just doesn't care.
From the article... "Scores of data-mining projects that collect and analyze U.S. citizens' personal information are in operation at dozens of federal agencies, the GAO found. Many of the nearly 200 projects planned or already under way rely on data purchased from the commercial sector"
They are purchasing the information from the commercial sector", information that is readily available to anyone willing to pay for it. Congress stopped their TIA initiative, but will allow this due to that major fact.
Hmmm.
Cue 200 posts about how terrible this is. It's not atually very different to what the law-enforcement community's been doing all along, just the tools are better.
It's like the argument about ID cards - there's nothing inherently wrong with being able to conclusively prove that you are who you say you are, but it's another thing entirely for people to be able to demand you prove the same.
'Don't worry' said the trees when they saw the axe coming, 'The handle is one of us.'
Bless you for posting this information. Quite an eye-opener. Good research.
Without laws preventing such mis-use, it runs rampant. European laws guarding personal information are much stronger than in the US, where corporate and government interests and methods are closely aligned, especially these days.
The game of Go (Igo, Weiqi, Baduk) has the simplest concept and the deepest play.
Like the Abu Ghraib Humiliation scandal the intellectual direction has been set at the top levels of the administration, even though the work on the projects is being done by lowly Primary Investigators.
What I wonder is whether the ones setting the intellectual agenda in the administration have any idea ofwhere they are leading us.
I fear, that one day we will be left in shock and awe when we discover that their idea of "preventing terrorism" was as well thought out as the idea of democratizing Iraq. We will all be at the mercy of a State that, like the Iraqi Monster, has grown too big for them to bully around. And then all of us, being in the same boat, will realize that those bas***** in the administration are screwed - but so are we.
.
To see a world in a grain of sand, and then to step back and see the beach where the sand lies
I think you have a 1 in 15 chance of clicking the link to RTFA... I clicked the last one... Doh, its a 71 page PDF! eek, better try another one
They want the massive database to fight "terrorism", as they say it. Now, lets just think about what's been going on lately;
1: Police officers have been taking pictures of protesters to throw into databases. Additionally, wearing masks is now illegal in most cities.
2: Facial recognition software enabled camera's are going up everywhere. Sure, the software barely works but it'll get better. This is first justified in protecting people in high crime areas like chicago, and then will move out as far into the suburbs as they can justify it.
3: Our economy is going to shit and we're going farther and farther into that hole.
4: Massive databases of personal information are being assembled by our goverment.
5: Our constitutional rights are gone.
6: And to put the decorations on the cake, at the G8 summit at Sea Island in Georga, an order to "fire to kill" was given to all police officers in the state in regards to whoever they found was acting suspiciously. This means that if you're a protester outside of the building, the cops and military in the area can legally shoot and kill you.
So, we've got our reichstag fire (9/11), we've got our Decree of the Reich president(Patriot act), now we're waiting for the enabling law (aka, the law that let the reichstag put people into concentration camps).
Candy-Coated Knowledge
Congrats! You're the last people on earth who trusts the Bush administration.
I figure that the government is so bad at effeciently using information anyway this doesn't matter. I don't know how this is a big deal compared to corporate data mining. Government is at least motivated by "serving the people" (yeah yeah Iknow that is nto always true) where corporate uses of data are motivated by serving shareholders interests (At least most of the time- *cough*Enron*cough*).
I tried for 5 years to come up with a clever sig...only to realize that I am not clever.
its a well rounded summary of a specific topic that includes links to back up nearly every point made _in_ the summary, click on the points you dont understand or would like to learn more about, skip over the rest and forget about it.
the poster has apparently gone to a fair ammount of trouble to bring these links together into a nice written summary so that you dont have to go out and search for the information yourself (as if you would anyway... pfft).
point being...
If you are too lazy to click on a few links to see the information, you should go back to the other room, turn on the TV sit down and shut the fuck up.
I can't agree less. The current administration has done a nearly flawless job so far.
Bush administration officials ARE responsible for the objectionable actions at Abu Gharib and ARE taking action against those that committed them. That's what being responsible means. Compare that to the previous administration that dodged most responsibility and lied to cover it up.
Don't poor-mouth the intellectual agenda. It is brilliant and is working.
I work for an electrical company in Florida and the local Police agency has access to our customer records whenever they want. Must make it easier for them to track people. - This also reminds me of the time I spent in a hotel for a few weeks. During my stay I noticed police cars in the front once in a while. I wonder if hotels share their customer info also. Hmmm. Makes ya think doesn't it?
This is why I'm working on a master's degree in data mining. Better to be on the inside. I for one welcome the chance to be your data mining overlord! Buahahaha!
Here is the EU Personal Data Directive (95/46/EC), which is the basis/minimum for the data protection laws of each EU and EEA country. The Norwegian Personal Data Act is supposedly the toughest of them all. It's interesting to compare the nuances.
People say I'm crazy, I got diamonds on the soles of my shoes...
Come on, are you a sheep? Are you going to rattle off a list of things like this and say you have no responsibility for changing things? If you honestly believe your list, you know the 2000 election was rigged and the 2004 election will have a similar result. So, politics are dead in the US. What are you going to do?
If you do nothing it will soon be too late. How about a repeal of the 22nd amendment so George W. Bush can be "President for Life"? Do you think this is too far fetched? Obviously not, from your list. So, if you wait until November to overthrow the government, it may be too late and we will have no choices left. At least now it would be possible to rebuild the electoral process.
I have trouble seeing how the availability of information is in itself a bad thing. Yes, it can be used to draw false conclusions, but that is a fault of its use, not its existence. And its not like false conclusions are never drawn without databases of public information. Anyone ever heard of racial profiling?
We should not worry about regulating the existence of such databases, we should instead regulate its use. Don't let the police get a search warrant based solely on data they found in a database, but allow them to query the databases to help them obtain the necessary evidence.
Mathematics is made of 50 percent formulas, 50 percent proofs, and 50 percent imagination.
Oh my dear lord, they're going to figure out that I buy computer books AND read _College Roomies from Hell_. And you know what that means!
/. Yeah.
Or maybe they'll be convinced I'm a terrorist because I read CNN *and*
Given that the G8 summit will be a huge target, what do you think they should do? Throw Twinkies and ask the nice man to please put the bomb down?
Balancing the right to protest with the prevention of a potential suicide bomber is tough. Where is your personal line? What would you have the security forces do?
And the saddest thing of course is that most people are outraged by the price of gasoline. We are indeed in sad shape. Bush should be impeached for lying about the Iraq "war". We really need to send Bush and his cast of kooks, Colin Powell, his son FCC chairman Powell, Condoleeza Rice, "Big" Dick Cheney, John Ashcroft, et. al. into early retirement. Their view of the way the world should be is too sick and twisted.
http://tinyurl.com/3t236
2. Data Quality Principle
Personal data should be relevant to the purposes for which they are to be used, and, to the extent necessary for those purposes, should be accurate, complete and kept up-to-date.
How does the government separate the junk data from what may actually be worth looking at?
People say I'm crazy, I got diamonds on the soles of my shoes...
3: Our economy is going to shit and we're going farther and farther into that hole.
GDP is growing faster. Unemployment is going down. Manufacturing is strong. The stock markets are moving up. Now by what measure does this translate to "shit"?
The facts are so clear the US economy is back on track again, why do these idiots keep sticking to their view from 3 years ago? This meme needs to die. It's not true, and it's certainly not insightful.
Slashdot Moderation: From positive to terrible in 2 "insightful" posts.
I quote : " And it is not allowed to give information away or even sell them (that is only allowed for public available information)". Lucky for you you have freedom of spitting untruth (no true freedom in germany) while being the real coward hiding behind anonymity... Only private data not available true a public source (like telephon book) is protected.
C. Sagan : A demon haunted world:
http://www.amazon.com/gp/product/0345409469/
visit randi.org
now we're waiting for the enabling law (aka, the law that let the reichstag put people into concentration camps).
You mean a law that would allow people to be detained without a reason, thrown in a Guantanamo Bay prison and not allowed to speak to anyone including legal counsel? Don't be silly that will never happen!
"So, we've got our reichstag fire (9/11), we've got our Decree of the Reich president(Patriot act), now we're waiting for the enabling law (aka, the law that let the reichstag put people into concentration camps)."
... the US already has its concentration camps ... and enabled the law ... Guantanamo Bay (have I written it right, not sure?)
You obviously have missed something
I am in favor of all datamining efforts that have an opportunity to inrease the security of the USA.
"3: Our economy is going to shit and we're going farther and farther into that hole.
... not everything that shines is Gold!
... and it translated to shit in 1939, when the economical breakdown was almost inevitable ... but hey, the solution was quite simple ... the germans just had to raid Poland ... hm, can't remember or was it Poland or was it Iraq with all the oil?
GDP is growing faster. Unemployment is going down. Manufacturing is strong. The stock markets are moving up. Now by what measure does this translate to "shit"?
"
Be careful
Since the grandparent post talks a lot of Nazi-Germany, I just want to add, that after Hitlers election in 1932 the German GDP was growing fast, too. Unemplyment went down, too. Manufacturing was strong, too. Stock markets moved up, too.
If you tell me that the 75 stories are the bottom stories then I agree with you. The problem is that you are showing me the top 75 stories standing up in thin air, and asking me to believe that the bottom 25 stories will soon be done. In this case, which is what it is with the Administration's "achievements" in Iraq, it is a cause of considerable worry. Pure and simple delusion.
.
To see a world in a grain of sand, and then to step back and see the beach where the sand lies
The trouble with info in databases is its persistence. You might forget an indiscretion of youth, but big iron never will. Anyone who's ever appeared in court and has been charged with a maximum allowable offense, later reduced to a much more minor offense, ought to verify that the lesser charge is on record. Bad records can ruin chances at employment. Many travelers to Canada find themselves blocked from entry, stranded at the border, because they didn't know their records were in error.
-----
We have had 4 straight quarters of record breaking growth
-----
Damn those slaves can pull hard when they're whipped heavily enough.
-----
and have added 1.2 Million jobs in the past 6 months
-----
Yeah, there are Wal-Marts and McDonald's are going up everywhere. I'm still suspicious of how those numbers are generated. If my company trims 5000 people and then hires 250 new people, is that 250 jobs added?
+++ATHZ 99:5:80
Ok, it all seems a bit weird to a European that the government can do stuff like that, but why do they need to.
Are we (in Europe) really crap at catching people or:
Do they want to catch everyone that makes the slightest error.
Is there a big problem in the US that requires data-mining compared to Europe.
Are you just crap at catching people in the US and need data-mining.
Are you planning on sacking half of the police force when data-mining by a couple of terminal operators catches everyone.
Help, it all seems like freaky big brother overkill, isn't a measure of civilisation 'the less you need to spy on your neighbour the more civilized you are'
thank God the internet isn't a human right.
This is not active privacy invading surveillance by the government. The is consolidating information from "publicly"/consented sources to look for trends. Consolidating legal and publicly available data is not invasion of privacy. These projects are forms of "passive surveillance", where we combine and consolidate information from a variety of already known sources to produce information that was not apparent before.
...o wait I don't have a tin hat on, and tin hats are in fashion right now on /.; time to watch this get modded down.
This is the government. We have laws, and a Constitution that (purportedly) protects us from the government by limiting its power through a series of checks and balances. Yes, this information may be available to the general public, but John Q. Public doesn't have military or police force that come beating in your front door and taking you into custody because your personal information matches a certain pattern - whatever that may be.
Herein lies, I'd argue, at least one reason behind the 4th Amendment to the U.S. Constitution. It basically says to the government, "If you don't have a reason to be looking at John Q. Public, you shouldn't be looking at John Q. Public." These projects constitute a massive fishing expedition, and we're all suspect.
First of all, in Europe, data is collected for specific purposes (use limitation principle) and can only be used for these specific purposes and not for other purposes, unless the data subject explicitly consents to new use.
Secondly, whether or not data is "legal" and "pubicly available," you have to get consent from the data subject in order to use that data. And of course you would still be bound by the use limitation principle.
"Passive surveillance" is not prudent, as the data collector risks collecting "junk data" (data that is incorrect, false, outdated, etc.), in addition to risking data subjects filing lawsuits.
People say I'm crazy, I got diamonds on the soles of my shoes...
Herein lies, I'd argue, at least one reason behind the 4th Amendment to the U.S. Constitution. It basically says to the government, "If you don't have a reason to be looking at John Q. Public, you shouldn't be looking at John Q. Public." These projects constitute a massive fishing expedition, and we're all suspect.
I appreciate the rationale behind your argument, but it is legally wrong. The Fourth Amendment says that the government may not conduct warrantless searches (except where obtaining a warrant first would be unreasonable). Of course, the antecedent question is, "Has there been a search?" And the Supreme Court has held that there is no "search" if you do not have a reasonable expectation of privacy (thus, a search of the garbage cans you leave out for collection is not really a "search" for Fourth Amendment purposes). So, as to information readily available to the John Q. Public, you have no reasonable expectation of privacy, and therefore the government may search it without running afoul of the Fourth Amendment because -- as a legal matter -- there is no "search."
So, having the most people working for the highest wages in history suddenly qualifies as slave labor?
Yeah, there are Wal-Marts and McDonald's are going up everywhere. I'm still suspicious of how those numbers are generated. If my company trims 5000 people and then hires 250 new people, is that 250 jobs added?
This isn't mystery science. The statistics are freely available to the public. For example, here are the statistics for the month of April:
The number of employed people went up by 278,000 to 138,576,000
The number of unemployed people went down by 188,000 to 8,164,000
The number of people in the workforce went up by 91,000, and the number of people not in the workforce also went up by 116,000
Compare this with the employment data from September 2003 and you can see that the number of employed people went up from 137,573,000 to 138,576,000, and the number of unemployed people went down from 8,973,000 to 8,164,000.
The average hourly earnings for production workers is at an all-time high of $15.59 per hour. Unless Wal-Mart and McDonalds have tripled their wages, this increase certainly isn't attributed to them.
-----
The number of unemployed people went down by 188,000 to 8,164,000
-----
Is that reported as newly filed unemployment claims? Even the nightly news will report that many people have decided that standing in line at the unemployment office isn't worth the effort.
All of the comments about the number of employed people correlates well with the new KFC they're building down the road.
-----
The average hourly earnings for production workers is at an all-time high of $15.59 per hour
-----
Averages are the most easily massaged numbers of all. Every statistician knows that. On a scale as large as the entire population of the US, an average number is marginally useless. An interesting number which would prove my point would be the number of people employed as "production workers" from September 2003 to current.
+++ATHZ 99:5:80
And I'd argue that this (even though it might be the current standing of the U.S. Supreme Court), has no merit, and the reason is simple: Our society is built on an infrastructure that, for all practical purposes requires that I do certain things. I have little choice but to place my trash out on a curb for pickup, because there are few, if any other practical options. The fact that I have few (if any) other practical options does not portend or in any way grant anyone a "right" to take advantage of something merely because it is available, and especially because its availability is a byproduct of the infrastructure itself. I use the infrastructure because, as a pragmatic and civil-minded denizen, I must.
Let's address the issue with the "publicly available" information. While there are certain records that are unquestionably public information, I'd argue that what I purchase is NOT public information- each purchase is a contract between myself and those vendors with which I choose to do business. Unfortunately, the easy availability, and wide use of credit, have allowed institutions to hijack, retain, sell, and otherwise use the specific details associated with each transaction, and make them available as they see fit.
Something to think about - Let's say I cut my finger while preparing dinner. I use a kleenex to clean the wound, before applying a bandaid. What am I now to do with the kleenex? It has my blood on it, and presumably, can serve as a "sample" for anyone interested in fishing it out of my garbage. What am I to do? I don't believe that anyone has has a right to acquire this without my permission, or without a warrant. I am placing it in the garbage not because I have any intention of making it available to the public, but because this is the proper and accepted method of disposal.
Here you can see the quandary imposed by a) the infrastructure, and b) the court's current standing. I don't believe it is fair, and I hope to see it change at some point in the future.
The GAO report that surveyed government departments to find out how much data mining they're doing and planning is interesting. In particular, the appendix that lists which departments don't do or plan to do any data mining has a few notable inclusions:
You'd expect some of those to have data mining as their primary function. Others are just hard to believe.
It'd be nice if it turned out that all these projects are intended to do things like:
- locate areas where increases in education, public health, and crime prevention spending would have a large positive impact
- identify major sources of pollution
- find patterns of police brutality
- predict and avoid major traffic problems
Somehow, I don't get the feeling that these are the goals of most of these projects.
Is that reported as newly filed unemployment claims? Even the nightly news will report that many people have decided that standing in line at the unemployment office isn't worth the effort. All of the comments about the number of employed people correlates well with the new KFC they're building down the road
No, that is the total number of people that collect unemployment. The BLS also tracks the number of people that drop off unemployment because they are discouraged about the job prospects. Guess what? That number is also decreasing (492,000 from a high of 514,000), and is only nominally higher than the 1996 figures when we had the same unemployment rate.
Averages are the most easily massaged numbers of all. Every statistician knows that. On a scale as large as the entire population of the US, an average number is marginally useless. An interesting number which would prove my point would be the number of people employed as "production workers" from September 2003 to current.
In other words, you refuse to believe the statistics because you simply disagree wtih them. All of the data you could possibly want is availble on the BLS website. The average weekly earnings for all private sector employees (which is plainly listed on the April report) is also at an all time high of $525.38. You can also see that the average work week for all private sector employees is steady at 33.7 hours/week, and the manufacturing work week has dropped to 40.3 hours/week. In other words, we are getting paid more for the same or less work now.
But this is pointless to show you any facts because you seem to be easily blinded by your hatred for the current administration.
When I give my personal info to another party, I retain the copyright on that info. That party is to use the info for only the express purpose for which I extended a limited license to copy it. When they give my copyrighted info to another party, they have violated my copyright. I revoke their limited rights, and require remedy for damages. This copyright protection applies to government, corporate and individual entities. I am protected from cross-agency access. You are, too. When will we take action against these copyright violations?
--
make install -not war
I wouldn't call it hatred. More "realistic". Every administration, including the current one, tends to skew the numbers in their favor especially in election years.
The average weekly earnings for private sector employment is $545.38? I barely make that with a top-notch education. Is it possible that number is skewed by the all-time high of CEO compensation disproportionately skewing the scale? $525/week, here in the US, barely pays the bills. Knock of 30% for taxes, then knock of 6% sales tax on any money that's spent and even a $600 rent, with $50 electric, and a $30 telephone, plus an average $450 food ($15/day), and $50 car insurance, plus at least $20/week for gas. It starts to get pretty thin on just basic living expenses. Heaven forbid anyone's paying college loans. We're in the red before we ever get out of the gate.
If I'm blinded by this supposed "hatred", you're blinded by a perfect naivete.
+++ATHZ 99:5:80
The govt's databases are nowhere near as fine-grained as the private sector's. Your spending habits, travel habits, job history, crime history - it's all there, more than the G-men could ever organize on their own.
This is warrantless search and seizure in my view. Looking for patterns to identify you as a potential criminal or terrorist? Is this what it's come to?
Big Daddy, Johnny, Burp, Aunt Zelda, Scott, Slurp, Big Momma
Gathering together databases of all your info and tracking it for habits that might identify you as a potential criminal or terrorist is a whole hell of a lot different than using those databases after the crime was committed.
Are we so terrified that crimes must now be prevented before they occur? Kryzt, give me a fuckin' break.
Big Daddy, Johnny, Burp, Aunt Zelda, Scott, Slurp, Big Momma
Easy answer: they don't.
I *hope* my data is useless and cluttered. Although security through obscurity doesn't work well, obscurity itself if better than nothing.
The more chaff in the air, the harder it is to see.
of course, who needs a law anyway if you're the US government?
The stock markets are moving up.
:)
Just what time period are you looking at? Let's look at the 5 year history of the DOW: http://finance.yahoo.com/q/bc?s=^DJI&t=5y
Through 1999 and until about June of 2001, it was hovering around 10,500 or so. Sept. 11, 2001 caused the dive to near 8000. DOW goes back up again to around 10,000 until about June 2002. Then another dive, this time down to about 7,200 in Sept. 2002. Goes back up to around 8,700 through X-Mas 2002, takes another dive through Q1 2003 (when we were gettin' our war on), then rises steadily to around 10,800 until it starts to flatten out around 10,100 starting in about Feb. 2004.
That's the short-term analysis -- a rather cyclical approach. Look at it long-term. Where were we 5 years ago? Around 10,800. Where are we now? 10,100, with no reasonable assurance that we are not headed for another dive in the coming months, particularly since the bond markets will strengthen once the Federal Reserve gets around to raising interest rates near the end of the summer.
IOW, it's been a rough ride for the DOW over the Bush Presidential administration's lifespan, and will continue to be through election day.
To quote a great and famous Republican who at least *tried* to cut government spending and waste, "Are you better off now than you were 4 years ago?"
No, we are not. And that's not only because the stock market is down from where it was when Clinton was in office.
Clinton screwed an intern; Bush screwed America. Vote Libertarian.
Is Capitalism Good for the Poor?