Homeland Security Department Testing "Pre-Crime" Detector

holy_calamity writes "New Scientist reports that the Department of Homeland Security recently tested something called Future Attribute Screening Technologies (FAST) — a battery of sensors that determine whether someone is a security threat from a distance. Sensors look at facial expressions, body heat and can measure pulse and breathing rate from a distance. In trials using 140 volunteers those told to act suspicious were detected with 'about 78% accuracy on mal-intent detection, and 80% on deception,' says a DHS spokesman."

Comments

Not even close

[ShawnCplus]

Sorry, but 78% is not even REMOTELY accurate to consider someone dangerous. There is already a high enough false accusation rate.

Re:Not even close

[JustinOpinion]

Cory Doctorow described it nicely in his recent book "Little Brother" (free download available):

If you ever decide to do something as stupid as build an automatic terrorism detector, here's a math lesson you need to learn first. It's called "the paradox of the false positive," and it's a doozy.

Say you have a new disease, called Super-AIDS. Only one in a million people gets Super-AIDS. You develop a test for Super-AIDS that's 99 percent accurate. I mean, 99 percent of the time, it gives the correct result -- true if the subject is infected, and false if the subject is healthy. You give the test to a million people.

One in a million people have Super-AIDS. One in a hundred people that you test will generate a "false positive" -- the test will say he has Super-AIDS even though he doesn't. That's what "99 percent accurate" means: one percent wrong.

What's one percent of one million?

1,000,000/100 = 10,000

One in a million people has Super-AIDS. If you test a million random people, you'll probably only find one case of real Super-AIDS. But your test won't identify one person as having Super-AIDS. It will identify 10,000 people as having it.

Your 99 percent accurate test will perform with 99.99 percent inaccuracy.

That's the paradox of the false positive. When you try to find something really rare, your test's accuracy has to match the rarity of the thing you're looking for. If you're trying to point at a single pixel on your screen, a sharp pencil is a good pointer: the pencil-tip is a lot smaller (more accurate) than the pixels. But a pencil-tip is no good at pointing at a single atom in your screen. For that, you need a pointer -- a test -- that's one atom wide or less at the tip.

This is the paradox of the false positive, and here's how it applies to terrorism:

Terrorists are really rare. In a city of twenty million like New York, there might be one or two terrorists. Maybe ten of them at the outside. 10/20,000,000 = 0.00005 percent. One twenty-thousandth of a percent.

That's pretty rare all right. Now, say you've got some software that can sift through all the bank-records, or toll-pass records, or public transit records, or phone-call records in the city and catch terrorists 99 percent of the time.

In a pool of twenty million people, a 99 percent accurate test will identify two hundred thousand people as being terrorists. But only ten of them are terrorists. To catch ten bad guys, you have to haul in and investigate two hundred thousand innocent people.

Guess what? Terrorism tests aren't anywhere close to 99 percent accurate. More like 60 percent accurate. Even 40 percent accurate, sometimes.

What this all meant was that the Department of Homeland Security had set itself up to fail badly. They were trying to spot incredibly rare events -- a person is a terrorist -- with inaccurate systems.

Fancy that, Burka's protect civil rights.

[tjstork]

If everyone was wearing a burka, then, there's no way that this system actually works. It may seem strange, but, what right does the public have to know my face?

Re:Err

[DriedClexler]

Yes, it does sound idiotic. My reaction was: ROFLcopter at the idea that you can successfully "tell people to act suspicious". Um, if it were possible in the first place for people to notice and control the aspects of themselves that make them look suspicious, others wouldn't be suspicious of those aspects in the first place!

Think about it: people become suspicious of others based on criteria X,Y,Z because meeting X,Y,Z reveals a higher probability of intent to cause harm. But anybody trying to cause harm will suppress any *controllable* sign that they are trying to cause harm before it's too late to stop. So the only remaining criteria people use in dermining whether they'll be suspicious of someone are those that are very difficult if not impossible to control. As a bad example: someone will only look around to see if he's being watched (which looks suspicious), if he's about to do something objectionable (like picking a lock). But he can't suppress that because then he takes the chance of someone noticing him picking the lock.

A better test would be to set up a scenario like a line at the airport where the screeners have to keep out dangerous items. Then, have a few of the participants try to smuggle items through, and get a huge reward if they succeed, while the screeners get the reward if smugglers don't succeed. Then, put a time limit on, so the screeners have to be judicious about who they check, so they only check the most suspicious. Oh, and make it double-blind as much as possible. Then, the people trying to smuggle will have the same incentive structure that real smugglers have, and thus will give off all the real-world signs of planning something objectionable.

But then, that would be too much work.