Slashdot Mirror
Data Mining In Law Enforcement
jcatcw points out a blog entry by Scott McPherson, CIO for the Florida House of Representatives. McPherson condemns the state of data sharing and data mining in law enforcement, saying that the US causes itself a great deal of trouble by focusing more on "antiterror armor and nuke-sniffing devices" than a useful information distribution network. He discusses a few such projects, and how they could have directly affected the events of 9/11. Quoting:
"One of those ingenious things that actually worked, Seisint founder Hank Asher's brilliant MATRIX system, remains mired in controversy and politics. Hank showed me MATRIX just a few short weeks after the 9/11 attacks. Using law enforcement data and commercial data, all of the commercial data available in the public domain, Asher's query produced [hijacker Mohamed] Atta's photo -- and about 80 others, many of them fellow 9/11 hijackers, many of them associates of the 9/11 hijackers. It was simple data mining and algorithms, and none of the information was obtained illegally."
so he managed to write some software that analyzed the internet - and managed to produce photos of some of the people that erm had already erm been identified. Surely (and maybe I've misunderstood something here) a 'result' would be identifying people likely to commit terrorist attacks, allowing enforcement agencies to monitor them and prevent them from commiting future attacks. (and no - this doesn't mean off-shoring every muslin who downloaded the Jolly Roger Cookbook).
"... obtained illegally"
As counter intuitive as it may seem at first, agencies have strict rules on this kind of behavior.
The Kruger Dunning explains most post on
Wow, really? You were able to identify after the fact? Great! Real useful -- that and the fact that it's much easier to find that information when you are looking for a specific result. If this guy had come out and said, "hey, I was able to find those people before the fact," then I'd be impressed.
That sounds like an adult version of my 8-year-old saying "I wuz gonna say that" when we watch Jeopardy as a family. It's real easy to come in after the fact and impress someone with results that are already determined.
I judt got a nre Kinesis keybiartf so please excusr ant egregiou typos.
TIA standing for total information awareness.
It was shut down over privacy profiling and other concerns, surely you remember, it wasn't that long ago.
This story seems to lament this but geeze, make up your mind, if it's not an outcry about the lack of datamining it's someone saying datamining is one foot in Orwell's 1984.
If this ever grows logs it'll become a political hot potato again and get dropped.
I keep watching the bar for spying on people get lower and lower.
First it was suspected enemy agentz.
Then it was suspected associates, even though separation may be 3-4 people away in a chain.
Now its anyone suspected of a crime.
How long until everyone is dumped in this database for not just intel or law enforcment, but potential employers, stalkers, and violent criminals data mining for easy marks?
It was simple data mining and algorithms, and none of the information was obtained illegally. 1. He doesn't tell us what the "Asher's query" was, leaving us with the impression that anyone could magically ask the right question and stop crime.
2. I wonder what he means by "commercial data available in the public domain". Either it's commercial and you have to pay for it, or it's public domain. My long distance calling patterns are commercial data (and is sold by the phone company for marketing), but they're not "public domain" in the way that most of us would understand it.
[Fuck Beta]
o0t!
It's like your 8-year old hearing the answer on TV - and then spieling off 78 random answers + the answer he's just been told.
I have a lot of issues with the various things in this article, but I'll keep it to one for now. Maybe Atta could have been arrested because of better coordination between local law enforcement. But his arrest almost certainly would NOT have prevented 9/11. Moussawi was supposed to be there that fateful day, and it still went down. One person arrested, even one of the many masterminds, would not have prevented it.
Also, no local law enforcement officer would have been able to piece together this plot from looking through one car BEFORE the event. Piloting multiple planes simultaneously into various landmarks was just too implausible to be believed before it happened. Even if John McClain himself figured it out, he wouldn't be able to convince anyone to help him stop 19 other people from boarding planes in multiple airports.
Sharing information sure beats what we're doing now, both in law enforcement and the intelligence community where I work, which is holding everything close so no one else can take credit. But let's not exaggerate the benefits here.
Tic-Tac-Toe, Global Thermonuclear War, and relationships all have the same winning move.
A few short weeks after the Kentucky Derby, I devised a database system that predicted the winner. Impressive, no?
I wrote parts of this stuff
"We didn't illegaly hack you, we violated the law legaly" *darts looks to both sides*
-- (this is a sig) My Computer Programming Forumhttp://www.programers.co.nr/
Data mining in law enforcement and anti-fraud will become more and more important in the future. Only recently computers are able to manage huge social networks. An often used technique is guild by associasion: your specific background, social network (who do you call), criminal record etc. give you a certain value. If this value is higher than a critical value, they will start further investigations. If you want to learn more about this subject I strongly recommend this videolecture of Foster Provost, a pioneer in this research: http://videolectures.net/mmdss07_provost_ilwn/
"It was simple data mining and algorithms, and none of the information was obtained illegally"
It's a shame more of the public doesn't realize that it's not necessary to either break the law or pass laws to legalize violations of one's rights, to provide reasonable protection for the public good.
I work for the Department of Redundancy Department.
You often hear about the police pulling over some guy for whatever reason and finding out he had an outstanding warrant or something. I've always wondered why they don't equip police cars with a video camera and the ability to OCR every single plate that comes into view. License plates all use the same font, so they should be easy to OCR, and in theory they use a high-visibility color scheme (though that's not always the case.) The camera would scan, read the characters, and compare it to a big list of stolen vehicles, stolen license plates, vehicles that fled accident scenes or other crimes, vehicles that belong to people that have warrants, Amber alerts, etc., and any "interesting" plates would pop up on the laptop that's now in most police cars.
I'm not saying it would put up a big "pull over and detain!" notice, but it could pop up the plate, the vehicle it should be on, the owner, and why it's of interest, then the officer would decide what to do. I.e., if a car pops up as belonging to a wanted 22-year-old male but it's obviously someone else in the car (too old, wrong gender, etc.) then they would ignore it.
Of course, like anything, there is the potential for abuse, but before you freak out about privacy, remember that driving, by definition, is a very public act. We're not talking about millimeter-wave radio or looking behind closed curtains with an infrared camera, we're talking about reading the required-by-law several-inch-high unique identifier on a hunk of steel with unobstructed windows on the public roads. If you're wanted and don't want to get caught, it's your responsibility to not go out in public with a visible unique identifier.
Dear Slashdot: next time you want to mess with the site, add a rich-text editor for comments.
This article reminded me of all of those 'psychic hotline' adverts on TV.
I never paid much attention to them because I figured that if they were really a 'psychic', then they would already KNOW to call me instead. Had to be some kind of phone charges scam I concluded.
Hmmm, maybe I'M psychic! (nah, I'm probably just psycho)
Down With Slashdot BETA!!! I've been around the corner and seen the oliphant; you can only abuse me from your perspecti
I've worked in the field of law enforcement data sharing. Fact is that most law enforcement agencies are either islands of automation or very loosely connected to other agencies. The stuff you see in TV and movies ("24") is a fantasy. Adjacent towns and cities rarely share information, and this lack of knowledge can put members of their police force in danger (for instance when making a traffic stop). A few years ago, the DOJ kicked off a sharing initiative with the Global Justice XML Data Model (GJXDM). This is an XML based specification for exchanging law enforcement data that was developed at Georgia Tech. I was involved in an initiative in Ohio to share police record management system information at a state level. The system was deployed and is operational today. GJXDM has been superseded by the National Information Exchange Model (NIEM). It should be noted that the NIEM model is even more complex than it's predecessor and tends to break many XML tools. The data exchanged tends to be fairly rudimentary and fairly sparse - arrests, bookings, warrants. Nevertheless, most agencies, and most states have either not implemented data sharing or are in the earliest stages of doing so.
[Insert pithy quote here]
I'd like to claim the information as my own research, but its all available on Wikipedia:
http://en.wikipedia.org/wiki/Compstat
Basically, this system instituted in the early 90s in NY was a major help in curbing NY's growing crime rate.
Just from 93 to 94 major felonies fell 12.3%. Of course this is not due to Compstat alone, but to a many programs independent, related to, and dependent on Compstat.
I vote yes on sharing on a broader implementation of this system - there are drawbacks I guess - but isnt there with everything?
Although datamining by organizations acting against freedom represents a huge threat, it seems to be the case that bureaucracy counteracts the efficient implementation of ideas, whether good or evil. Even the jerks who want to reduce us all to unthinking data-points under constant surveillance and targeted marketing are thwarted by the tendency of management to be counterproductive, regardless of the aim. Look at the attempts to upgrade the IT infrastructure of the IRS, FBI, etc. Even such a simple thing ends up years over schedule and billions of dollars over the original budget before being scrapped. Look at the contract awarded to Boeing for those giant surveillance towers to guard the southern border of the US. They totally screwed up and had to admit defeat, over budget and years after it was supposed to be rolled out. They try to do bad things on a large scale, but they're just too stupid.
"Asher's query produced [hijacker Mohamed] Atta's photo -- and about 80 others, many of them fellow 9/11 hijackers, many of them associates of the 9/11 hijackers." What he didn't say was what would have happened to the other, non-criminal names on the list. I suspect that they would have been stigmatised, harrassed or sacrificed to ensure the safety of those in power. I'm always a bit dubious of the effect of being a false positive in this sort of system.
Identifying the 9/11 hijackers was just an exercise. The next query will give them Morpheus, Neo, and the passwords to the Zion mainframe.
"Data Mining In Law Enforcement"
I'll take "How do you round up the most possible innocent people and make false charges against them" for $500, Alex...
If you can read this, I forgot to post anonymously.
The same techniques will likely be effective for identifying most effective protestors against current administration, or people that can be most effectively exploiting sexually, financially or politically. In fact, terrorists generally cover their tracks much better than innocent civilians.
This guy also doesn't seem to have much knowledge of intel gathering. The idea that forward projection isn't happening is...uh...wrong, and that's all I'll say on the matter (disclaimer: I'm ex-NSA)
If you're ex-NSA, then you also know that the difficulty isn't in writing the algorithms, it's in getting somebody to stitch together all the goddamn databases that are strung out all over creation.
Shit, *I* can write the social networking algorithms, anomaly detection, etc. But it doesn't do any good if you don't have the data integrated, and despite what's happened the last 8 years we still don't have it.
I also don't get the false dichotomy the author uses to rag on sensor-based detection.
nuff said.
-- Will program for bandwidth
I'm throwing around some hip "technical" terms and how applying that nerdy stuff could've helped 9/11. Because everything is about 9/11. Do you feel terrorized yet?
Give me money.
and its called ANPR
http://en.wikipedia.org/wiki/ANPR
cars,bridges,tunnels
I thought only the United States Government
Hank showed me MATRIX just a few short weeks after the 9/11 attacks. Using law enforcement data and commercial data, all of the commercial data available in the public domain, Asher's query produced Atta's photo -- and about 80 others, many of them fellow 9/11 hijackers, many of them associates of the 9/11 hijackers.
Without additional information it's impossible to say if this is impressive, or just a stupid algorithm trick. With many mining algos, you can easily train them pull certain needles out of the haystack. The question is, will your training situation look anything like the future situations? Training the algo only with the 9/11 terrorists, would it pull out the trade center bombers, or Timothy McVeigh? Will future predictions be right or will it pull out groups of Arabic student pilots who had the misfortune of buying the same shampoo most preferred by 9 out of 10 terrorists. Especially with rare events, I think you mostly get into a hyper complicated version of correlation != causation.
 Hank showed me MATRIX just a few short weeks after the 9/11 attacks. Using law enforcement data and commercial data, all of the commercial data available in the public domain, Asher's query produced [hijacker Mohamed] Atta's photo -- and about 80 others, many of them fellow 9/11 hijackers, many of them associates of the 9/11 hijackers. It was simple data mining and algorithms, and none of the information was obtained illegally."
I smell bullshit... think they did know before hand and are just saying this now to scare people into accepting complete (spying) monitoring of all net userÅ
and none of the information was obtained illegally.
Well I guess that depends on who you ask. The State is Michigan was sued by a former governor a forced to withdraw from MATRIX. I am happy to say I am no longer in the Matrix.
I do web data mining for a living and there is no way any algorithm or a combination of them can give you that kind of accuracy. You will have to be a few light years ahead of current published research to do that. Unless of course the system is drawing from published news about the suspected terrorist attacks in which case what they did was do-able (not as easy as one might naively think... the web is a pretty dirty medium but definitely do-able). I will believe that kind of a thing when I see it.
The US does a lot of data mining. In march Germany and the US signed a treaty under which, the US gets information about the political opinions, race, religious beliefs, sexual orientation and membership in a labour union of people from Germany. This is not the first time those information were collected in Germany. Last time the nazis used those information to bring jews, communist, socialist, sinti and roma or homosexual to the concentration camps
Data mining's actually old school these days. We're at the next step. Not just looking at data and using IT resource to query data, but systems which look at the relationships between objects in the data and which can present networks of interest to non-IT-savvy users. And that's the key, the people doing data mining are IT experts, not criminal or terrorist experts and that causes problems.
Check out NetReveal and I think there's also another product on the horizon by some other company.
Pinky: "What are we going to do tomorrow night Brain?"
Brain: "I would tell you Pinky but this 120 char limi
FYP.
I am glad to hear that the government is putting Data's lost head to good use... lets just hope they put his head back in the cave where they found it so that when the Enterprise finds it in the future, they will still be able to reattach it. You know, to help keep the timeline in tact. One can never forget about the Temperal Prime Directive. Plus, we wouldn't want Picard to die.... Data did a good thing by taking his life for Picards... we owe him in the current to not forget to put his head back where we found it.
WWPD - What Would Picard Do?
I just had a quick grep on Google for 'Hank Asher'. The results were spectacular - in fact I think I should tip off the 9/11 'it was explosions and holograms' crowd!
I always did wonder how they managed to come up with 19 names shortly after the original 18 was proven as bogus. Maybe we are at war with make-believe MATRIX instead of make-believe al-qaeda, or perhaps they are both one and the same. (alqaeda == the database, al-qaeda is widely acknowledged as invented by the government so they could prosecute UBL for being part of a terrorist organisation, presumably because there is no evidence beyond hearsay linking him to the embassy bombings of '98.)
A sample story from the Google:
Another Friend of Giuliani's Embroiled in Sheriff's Criminal Case â" Giuliani's Partner Linked to Bribery Case 04 Dec 2007
When Hank Asher reached into the bag and pulled out the two $15,000 gold Cartier watches, the holiday crowd at Carmine's restaurant on 44th Street in Manhattan noticed, patrons recalled. Later, so did the U.S. attorney in Orange County, Calif., and soon yet another of Rudy Giuliani's business partners was embroiled in a bribery case. Asher, identified by the initials H.A. in Overt Act 59 of a federal grand jury indictment against Orange County sheriff Michael Carona, had handed the diamond-encrusted Cartier baubles to the wives of the sheriff and his deputy, and with that, assured himself a place in a federal indictment that was looming.
The full article and your own research is highly recommended:
abcnews.go.com/Blotter/story?id=3952112
The ACLU produced a briefing on this a while back, in pdf. I converted it to html. Here it is:
http://thewall.civiblog.org/rsf/aclu_matrix_report.html
It's part of a downloadable collection of documents relating to Warrantless Domestic Surveillance I put together.
May You Find It Useful.
-dcm
It's not hard to imagine that money talk$ and bull$hit walks. And so, anyone...anyone with a database may sell that info to law enforcement....or simply to a marketing agency...who then sells it to law enforcement.
So, beware of what songs you place on your iPod. Everytime you re-sync to the mothership it could be firing off a little XML message that, when scatter-plotted against your DVD purchases, travel iteneraries, phone records and penchant for black clothing and running shoes...
You might be a superhero! Or worse.
Why nuke terrorists and not career criminals?
I think the answer is pretty clear. The policy makers have decided that a nuke incident, say North Korea smuggles a 10KTon weapon into the USA and sets it off in lower Manhattan (NYC). That, according to the Fascist Oligarchy that runs the country is of greater significance, greater disruption, greater impact upon the Fascist Oligarchy than would be a career crime family extorting, drug running, running gambling operations, killing a few citizens for non-compliance. You simply cannot hide a 10KTon nuclear burst in lower Manhattan. You can hide organized crime by having your toady main stream media not report on the career criminal crime family's operations.
That's my story and I'm sticking to it. Of course, YMMV.
Have gnu, will travel.
http://en.wikipedia.org/wiki/Hank_Asher
read who he is!!!!!!!!!!!!
Dear A/C, please read the article before diving off of the deep end:
...and my comment disclaimer:
"Asher's query produced [hijacker Mohamed] Atta's photo -- and about 80 others, many of them fellow 9/11 hijackers, many of them associates of the 9/11 hijackers."
"in fact I think I should tip off the 9/11 'it was explosions and holograms' crowd!"
Yep, I believe the 9/11 'truth movement' have lost the plot, brains fried from too much paranoid conspiracy. That said, a lot of people in New York and elsewhere believe that the government were either behind the attacks or that they knew about it and let it happen on purpose (LIHOP).
In this latter scenario another previously unimagined group could have been behind the attacks (maybe a U.S. based McVeigh-style outfit, lurking in darkest Big-Sky country or in Boston bedsits). Obviously the government would not have been able to get their PNAC war on terror if the culprits lived in the USA or another NATO country, hence, if a domestic group were involved, the blame would have to be elsewhere with make-believe suspects framed for that purpose.
Those not happy with the official narrative have had exactly as much time as you have had to think about what-happen yet they did not believe the election-buying, warmongering-crooks that you fell for, seemingly hook, line and sinker. The 9/11 'doubting Thomas's' can say things that don't stack up, but, to my knowledge, none dispute that there once was a twin-towered WTC. (Your assertion?)
Okay, this was not the first time that the authorities went after the wrong suspects (e.g. the first WTC bombing incident), however, as of yet, no explanation has been provided as to why the original list of 18 had to be changed to 19 hijackers. At the time the world was so shocked by what had been seen that nobody dared ask basic questions, like how a new and definitive list could be produced so quickly or what the supporting evidence was.
This article reveals a mechanism for quickly producing a (near identical) list to that foisted on the world by the feds, with the database developer demonstrating the capabilities somewhat boastfully. Given the incorrect first list and the absence of any other plausible detective work used to get the names right, I think it is about time that the freedom-of-information request goes in to find out if MATRIX was used to produce the suspect lists. If that is the case then some serious questions need to be asked regarding MATRIX and the vested interests behind it.
Note that nobody on this board thinks MATRIX is credible, so if it was used for 9/11 then the lists of suspects are 'incredible' and the trillion-dollar war is further undermined - sorry about that...
Finally, here's a 'lil quote from the 'Big Lie Aryan' that had finance help from grand-daddy Bush (before having to suicide himself when the WWII chapter came to an end):
"Naturally the common people don't want war; neither in Russia, nor in England, nor in America, nor in Germany. That is understood. But after all, it is the leaders of the country who determine policy, and it is always a simple matter to drag the people along, whether it is a democracy, or a fascist dictatorship, or a parliament, or a communist dictatorship. Voice or no voice, the people can always be brought to the bidding of the leaders. That is easy. All you have to do is to tell them they are being attacked, and denounce the pacifists for lack of patriotism and exposing the country to danger. It works the same in any country."
"Sorry, I should have reversed the owner/driver"
Wouldn't make any difference, it would still be ridiculous hyperbole.
That's it's slightly more realistic, and supported by a worthless anecdote doesn't make it less ridiculous.
Read that until you get it.
WOW! You FINALLY got it!
I don't want to discuss, you're full of shit and I want to tell you so. What you think on the subject doesn't mean a damned thing to me.