Slashdot Mirror
DHS Chief Wants Better Algorithms For Analyzing Intelligence Data
coondoggie writes "Better algorithms to spot patterns and trends within the mass of information the Department of Homeland Security sees everyday are key to national security. That was but one of the talking points DHS chief Janet Napolitano focused on in a lecture on the role of science and technology at the Massachusetts Institute of Technology today. 'DHS is part of the nation's Intelligence Community, which receives more terabytes of data each day than the entire text holdings of the Library of Congress. The National Counterterrorism Center's 24-hour Operations Center receives 8,000 to 10,000 pieces of counterterrorist information every day. We receive data about all of this, and it is clearly too much to suggest that the simplistic "connect the dots" analogy accurately represents what an analyst must do. Very quickly, you can see that "Big Data" – more so than the lack of data – becomes the most pressing problem. At the same time, the threats implicated by the data are not static.'"
More like he wants to be able to sift through more people data and make files on every citizen faster, am I right?
So you want to know how to turn terabytes of naked pictures of little old ladies and children from your full body scanners into actionable intelligence on Islamic radicals and wikileaks contributes? If you actually get some software, let me know... because whoever comes up with it has got to be the coding Messiah.
Well, if the DHS Chief wants better algorithms for analyzing intelligence data, maybe he can have the department's funding cut, so more money can be put to use educating a generation of better students.
-I only code in BASIC.-
So they have heaps and heaps of data and now find that they can't really make anything out of it. But they'll keep the data because who knows, maybe someday they'll figure it out. Or maybe just for bragging rights. That's what being in a TLAgency is all about, isn't it?
So they cast a net so large the boat sinks from the weight of the fish... Typical DHS.
Eating the brains of your enemies does not make you smarter. But it's still fun.
Step One: cut down on the clutter. That means stop chasing after everyone in a turban or every angry blogger named Mohamed or Assad who criticizes your government or sticking GPS monitoring devices under their cars.
There are two major reasons we've historically said NO to dragnets. They violate civil rights and they...surprise surprise!...clutter your intelligence channels with worthless drivel.
I swear to God...I swear to God! That is NOT how you treat your human!
pony too!
policy after policy is implementing her dream of a police state. traffic revenue cameras in Arizona when she was there. Naked scanners for the tsa. Pretty sure she is abusing patriot act powers. US Citizens need to take back their government
The DHS has far too much funding if they have too much data and can't analyze it with current algorithms. And really, why do they need all this data in the first place? Chances are the vast majority of it is unconstitutional and shouldn't be collected to begin with.
Taxation is legalized theft, no more, no less.
Join today, and help us build a better police state! Think of the children!
Perhaps the few people that read this and agree could grow a conscience and stop helping them out and keep them from hurting people.
You mean TV lied to me again and you can't foil terrorist plots by throwing data at a big shiny computer with lots of flashy light after all?
What with monitoring all the nation's phone calls, that is a lot of data to sort through. Listening for keywords probably isn't enough.
And yeah, I really do think they have automated systems that (attempt to) monitor all the phone calls made within the United States, as well as the more obvious targets of those going out from the US or coming in to the US. I can see of no other reason for some of the legislation they've pushed for these past several years.
#DeleteChrome
The PATRIOT act is an abuse of powers because it gives the government powers that are explicitly denied to them in the constitution.
And we won't see US citizens taking back the government anytime soon, since all of them have become so dependent on the government for all their entitlements.
Taxation is legalized theft, no more, no less.
I'm sure this will be a big help in identifying and prosecuting copyright infringements.
They should take a look at google's big querry.
PI
Questions raise, answers kill. Raise questions to stay alive.
We were working on some cool technology applicable to just this sort of problem back at a previous employer. Then 9/11 hit, management realized that the coming intelligence applications of this would put it on the ITAR list. Everything went into boxes and went offshore.
The next time you wonder how credit card, telecom and various other marketing concerns can find you, find your friends and figure out who is likely to buy what, you'll know who to thank. There's more money in pushing junk at you than keeping you from being blown up.
Have gnu, will travel.
The problem isn't finding the patterns. Finding patterns is easy. Eliminating the majority of patterns which have no significant meaning, that's the hard part.
...not Big Brother slowly creeping upon us like in the novel "1984"? I guess if you institute it slowly enough, people just bend over and take it.
"since all of them have become so dependent on the government..."
The point of (almost!) no return actually requires only more than half of them, nominally! BUT I still hope that the minority of liberty-loving people (both on antiwar Left and true conservative Right) can organize themselves better (without needing any "Community Organizers" :) ), and that there is still a chance to turn things around.
But then, I have my signature, and you have yours, and neither changed the world just yet... ;-/
Paul B.
post what you have on /. and we'll tell you the patterns. Your data will be perfectly secure here, no-one ever reads the articles anyway.
Nullius in verba
In japan a nuclear disaster is going on, and the jewish slashdot.org is keeping nerds busy with more and more nonsense.
One side consequence of better processing will be that it will be more advantageous to collect even more data.
Didn't lots of people say data mining for terrorism/"security" was ineffective?
looks like the native chiefs were much better with that stuff (weather,
spotting the bad guys, star gazing & hand waving, real stuff (until we
showed up?), than we are?
monitoring endless moronic/off-topic chatter to find something of value?
sounds like our legislative loop? eavesdropping must feel like being
chained to someone else's fear?
just whois plotting our next shitslide into hell? if we knew who it
was/is? looks like folks would prefer dying to being tortured nowadays. is
that new?
suck this in on the wire: we're more than finished with murder & mayhem .5 billionerror
inc... you might think about snooping in on those eugenatic vaccine
terrorists, who have damaged/killed numerous bips. & those
bad 'math' baal worshippers? yikes. we know everything's a big fauxking
secret, so we won't expect to hear anything until the (federal/world
court) trials begin. & all those weapons peddlers in lear jets/yachts?
smells like danger there too? lot of bips being mowed down in the desert.
sheesh. thanks.
DHS acts like the proverbial drunk searching for his dropped keys under a lamppost despite having dropped them in a darkened alleyway. Rather than scanning every word passed over electronic media they should have agents in the field doing investigations and talking to people who may have information. There is no substitute for human intelligence ("intelligence" as in "intelligence collection", although it appears that DHS has neither form of the term in excess).
So I suggest that DHS get off their collective fat asses, sitting in front of terminals, drinking Starbuck's lattes, and looking up their neighbor's wife's bra size and instead learn some useful foreign languages, circulate among foreigners and domestic groups of interest, and get to the source of the problem.
Despite the fact that this was started under a Republican President with Republican majorities in both houses and that it's primarily Republicans agitating for more and more invasive measures? Sure the Democrats have managed to fuck this up pretty good, but let's be honest about when and how this started.
sounds like a job for Watson - or at least a slightly modified version.
http://en.wikipedia.org/wiki/Watson_(artificial_intelligence_software)
I guess they could re-use that hand which is currently used to press the buzzer to, say, feel your balls when you opt out of the x-ray.
In all seriousness. DHS would do us all a whole better if they concentrated on conventional "police work" investigations, and less on fancy shit like warrantless wiretaps and X-ray backscatter arrays that don't work.
Our security apparatus had all the details they needed to make the connection before, and failed to see it not because they lacked some fancy algorithm, but people "in charge" didn't listen to the street operatives, and political infighting between organizations: which (hey, guess what Interplanet Janet!) DHS' very existance was supposed to solve.
So what she's saying is, "we suck, give us more money." Sorry, fix what you have first, Janet.
It MUST be pulled out, BY THE ROOT!
"Flyin' in just a sweet place,
Never been known to fail..."
Here's your algorithm, I even implemented it in code for you
/* We return a float (hey, not like we need precision) between 0.0 and 1.0 representing threat because we like to keep things one dimensional so we don't have to think too hard. */ // We assume threat is always there anyway, look harder
float evaluateIntelDataSet(t_ids *inteldataset, int count) {
for(int i=0; i<count; i++) {
inteldataset[i]->doAbsolutelyNothingButWasteTime();
}
return 0.1;
}
freedomboxfoundation.org
More than just a few years ago, all the spook houses in the US (and a lot of other places) yelped long and loud about Total Information Awareness (tm). Oh my we were told, every last bit, byte and sniff are available for scrutiny by all and sundry. Big Brother is watching YOU! Britain has 10 million cameras looking at everything in all public squares, Carnivore, Omnivore, and Echelon are all watching you. Every satellite signal, every cell phone signal, every email and Slashdot rant by the crazed nerd-o-philes gets the once over by the guys behind the grey one-way glass. Teeny problem. TIA means you have to analyse all that crap. Gobs of it. Yottabytes of it. So its all there. Every needle about every terrorist group, UFO invasion, and cancer cure is all there. In that great big ginormous haystack. If only we could find the needle, or set of needles that actually means something. We have outsourced any jobs that involve high technology. In the United States right now, 10,000 people who could help wade through the data are pumping gas or doing some other work because there is no job available for them. Old Janets problem won't go away. Its an American problem. Go ask a dumb suit like Aaron Barr formerly of HBGaryFederal to do some statistics for you! Oh yeah, he clearly failed stats. But look at that winning smile, firm handshake, power tie! Clearly he can make up a bunch of terrorists for the US to chase after, you know, just like the Iraq invasion!
Likely Janet "Planet" Neopolitano in her cliff notes wrote, "Al Gor Rythem."
But yes, better algorithms are needed and big data is a problem.
In that case, dissastablish DHS, in particular, and give walking papers to Janet's lap dog Pistole, and use a 10th of the money to fund math departments at US Universities and Colleges.
A no brainer even for Janat Planet, who does not have a brain. Pity Pity Pity. Oh ... DHS got screwed to the tune of several tens of million dollars by a Drunk from Las Vagas on "Algorithms."
-308
There won't be a revolution. America has televisions. It's just so much easier to grumble about the government online than it is to actually do something that matters.
If there's one fucking thing we've learned recently, it's that law enforcement types are god awful fucking gullible when they think they're listening to their own kind. We've hard proof that the Egyptian secret police carried out two major terrorists attacks inside Egypt and blamed their own country. And western intelligence services just blindly believed their lies. We understand how the evidence for chemical weapons in Iraq was entirely & purely focally fabricated by Iraqi ex-pat living in Germany. We even had some asshat put his wife on the no fly list when she was outside the country.
We obviously cannot trust our law enforcement and intelligence services without serious judicial oversight. In particular, we cannot trust the buffoons hoping that 'better algorithm' will give them some pre-crime detector. Instead, they'll just harass more innocent people based upon imaginary plots and connections.
The Christian religion has been and still is the principal enemy of moral progress in the world. -- Bertrand Russell
Clearly they are suffering from a diseased paradigm, which could be easily mitigated by a proper does of mind expanding ergot. No algorithms necessary; just the right perspective.
Just don't gather so much data. That makes for a friendlier community.
There won't be a revolution. America has televisions.
Egypt has televisions too.
Artificial Intelligence is no match for natural stupidity
Mohamed and Assad the blogger are easier targets than groups like CAIR with their vast funding from foreign sources and media exposure. The Islamic community in the US is a target-rich environment for real counter-terrorism and intelligence operations because there are so many Saudi and Iranian-funded groups of dubious or outright illegality working behind the scenes.
It's not even the dude screaming "Allah Akbar" and "Death to Israel and the US" on Youtube that should scare anyone. It's the guys in the suits, working for the NGOs and "charities" that are fronts for radical Islamic groups in the Middle East. Every moderate Muslim writing on the subject I've seen points out that the commanding heights of the American Islamic community are increasingly controlled by these foreign sources.
The tragedy of this situation is that liberals are attacking Peter King who is the only member of Congress with the stones to even raise real questions about the radicalization. The left is living in denial by blaming it exclusively on its usual hobbyhorses: poverty, discrimination and US foreign policy. While those are certainly factors, the left is openly in denial about the fact that there are powerful groups in the Middle East that want to radicalize the US Muslim community and turn it into a weapon against the US (which causes suffering to Americans, Muslim and non-Muslim).
I think she should do the job herself.
Perhaps it would be better to focus on intelligence gathering that requires them to leave their comfortable offices.
Sitting and essentially browsing the data of the internet is a nice way to spend the day, but I know as well as anyone that you don't get a lot of work done.
Some of us are quite clear on when and how this started. Please dispel your own delusions.
"Somebody has to do something. It's just incredibly pathetic it has to be us."
--- Jerry Garcia
Post it on the internet, I am sure Google will be more than happy to index all your information for you.
Violence is the last refuge of the incompetent. -- Isaac Asimov
Say a few "scary" words and my government listens in on my phone conversation. Then start talking about what I think the problems are in the country and about what I think should be done to fix them. This is a lot more convenient and cheaper than paying a $1000.00 to have dinner with my congress-person when they are looking for money to be re-elected.
Hey, wait! Are they trying to filter me out? Do I have to update my algorithm?
Department of Homeland Security actually getting something done? Oh dear god no.
When she says "counter terrorist information" she means "your private communication"
Napolitano is just one more of the long like of elected idiots who understand neither technology or their
job, which is (military) Intelegence and are trying to use computers in exactly the wrong way. Empirical
Computer modeling and 'tell-me-what-is-going-on' are beyond modern technology.
Hunches and good intuition is presently reserved to the people, and all this anti-profiling, security theatre
and techno babble just gets in the way of having a small team of expert investigators in FBI and CIA and
also the national intellegence agencies of UK and Europe eg DSGE, BND. these staff need to work directly
to their director and have direct and immediate access to the Sec HS and Justice.
The need all to be arabic speaking and knowledgeable in the Quran and Hadit, they need to have had
field experience.
A course with ShinBet wouldn't hurt.
Gee, maybe they just shouldn't collect so much? Rather than shotgun everything, why are they not being more judicious with what it is that they are monitoring?
Pointy Haired Bureau
Proverbs 21:19
its very interesting And thought provoking