Facial Recognition Fails in Boston, Too

bryan writes "Only a few weeks after cameras were found to be ineffective in catching criminals in Tampa, FL, a test of a facial-recognition system in Boston's Logan airport also came up disappointing. The cameras which were given photos of employees to detect, were only successful in 153 out of 249 random tests over the past year (about 61%). The article did not say how many false positives the tests generated. The companies involved were Indentix and Visage."

Comments

The system would work better...

[MikeCapone]

...if the cameras were mounted on black helicopters.

Geez, you people could at least read the headline!

[Prince_Ali]

It has 39% false negatives, not false positives!

Re:Face recognition

[boinger]

I think you have quite a misunderstanding of the difference between "false negative" (which is what the 61% was referring to) and "false positive" (which was not mentioned).

wetware comparison

[Empiric]

I heard that a very substantial amount of our brain's capacity is devoted to differentiating faces, and it's conjectured that this processing overkill is responsible for such things as people seeing a "face" in the objectively very non-face-like features of the moon.

Give the parallel processing capability people have to do this trick, it's probably not too surprising that computer tech hasn't gotten there yet.

Anyone know more about face-recognition processing in the human brain? I find this topic quite interesting...

Re:wetware comparison

[Hettch]

I've only had a breeze over the subject in one of my courses, but it astonished me. Our professor told is a story of a girl who head a head injury and actually lost the part or her brain that did the facial recognition processing. The description she gave was that of people looking normal from the neck down, but their faces were blurred almost like what they do to censor people in TV.

The facial recognition part of the brain is also very responsible for driving emotions. You show a picture of Hitler to someone, and they automatically become angry. You show a pretty girl, and they become more happy. A fascinating subject, its no wonder the cameras failed.

Re:wetware comparison

[bytesmythe]

I can add a little to this...

I wouldn't call our system "overkill". Also, there are really multiple systems involved.

First, there is the subsystem that recognizes a face as being a face. There are certain clusters of neurons that fire in response to any face-like pattern, regardless of whether or not it is actually attached to a head. This is how we recognize animal faces as being faces, the man in the moon, smiley faces and emoticons, Jesus in a water stain, etc. This capacity is innate, and infants can discern face-like patterns very soon after being born.

After a face has been perceived, it must be narrowed down to an individual person. This ability is partly learned over time, and is responsible for the difficulty people have in recognizing faces outside of their own cultural group. Certain types of brain damage (from a stroke, for instance) can allow people to recognize the fact that they're looking at a face, but still be unable to determine whose face they're looking at.

Keep in mind that even before a face is perceived, you have systems that find the basic shape outlines, determine their orientations (separately and with respect to each other), and at the same time attach color information, shuttle it off to "face subprocessing", then call up any related emotional context (have you ever seen a stranger you didn't like because they resemble someone you already dislike?) -- all before you can become consciously aware of the face.

Re:wetware comparison

[wirelessbuzzers]

Caveat: following is IIRC; this is from a basic developmental psych class that I took for breadth requirements. I got a lousy grade in the class mostly for failing to remember specific details, so some of this may be just plain wrong. Go to Scirus or something and search for this stuff before using it in your doctoral thesis.

Human face recognition is also built-in. Psych experiments on newborns (straight out of the maternity ward), as well as older infants, indicate that they can detect faces early on. This is not the case with all visual abilities that people have! There are many types of spacial recognition and object-parsing tasks that infants, and even toddlers, simply can't do.

Newborns pay more attention to shapes that look like a face > over those that are schematically similar > over those those with eyes, noses etc but in different arrangements > over ovals with random junk > over blank ovals...

Also at an early age (don't remember when; I don't think they tested it on newborns, but I wouldn't be surprised if they can do this too), babies can tell familiar faces from unfamiliar ones, and show an inverted habituation effect; that is, they prefer to look at familiar faces than unfamiliar ones (unlike most shapes, where they get bored with the familiar ones).

Also at an early age (again, I don't remember how early, but less than a year), infants map others' faces onto their own and imitate. That is, if you stand in front of them for a few minutes with your tongue sticking out, they often stick theirs out too. If you have one eye open and one closed, they'll copy that too (I don't remember which eye tho).

From an early age, babies can also follow gazes to tell what someone is looking at; this is important in the development of language as well as vision, because babies use it to figure out what an adult is talking about. It is, IIRC, used more than what the adult is *pointing* at.

In addition to recognizing faces, babies can recognise other body parts, and treat an action differently based on whether it is done by a hand or a stick (when they don't see the hand holding the stick). If it is done by a hand or other object perceived to be animate, it is treated as goal-oriented and categorized partly by the perceived goal; if done by an "inanimate" object, the baby does not look for a goal. This is studied through habituation experiments; different actions with the same goal were seen as more similar (and hence less interesting) than those with the same basic appearance, if and only if they were performed by an animate-looking object such as a hand.

Re:wetware comparison

[greenhide]

Not susceptible to any sort of bias?!?

May I direct you to the following quote, from a highly notable artificial intelligence program:

I hate this place, this zoo, this prison, this reality, whatever you want to call it. I can't stand it anymore. It's the smell, if there is such a thing. I can taste your stink, and every time I do, I fear I've somehow been contaminated by it.

Proof!

Computers hate us to their very bones. My computer has only crashed when I've been doing something important, like writing a term paper or surfing for porn. They're out to get us, all right.

Improbable to start with

[lawaetf1]

As super-duper as high-tech is, I think even /.'ers would admit that its not a panacea (yet) for all our security ills. The very idea of having a computer capable of accurately identifying one face in thousands -- scanning from afar -- is far fetched. Despite billions in research we've yet to master voice recognition which is, comparatively, much easier to do. Ah well, what's another few hundred million of tax payer's money shot. I'm sure it made some contractor rich.

Re:Improbable to start with

[krymsin01]

Yeah, but you have a government who is willing to spen the taxpayers money on this sort of thing. By and large, most taxpayers do not care about their privacy being taken away from them under the guise of security. Even if they did, you would think that more than the less than half of the population that votes would actualy vote to stop it.

As for it not being there yet, a lot of people said it was a far fetched idea for the US to send people to the moon, and in fact, a few people still believe that it didn't happen and it couldn't have. I'm willing to accept that it did happen, because the US Government wanted to show up the Russians and beat them to it. They were willing to spend the money, the technology emerges. Same thing here. If the government wants the tech, all they have to do is throw money at it, and wait. It'll eventualy be here before you know it.

Point of view

[mattdm]

I'm not sure I would call the failure of big-brother tech "disappointing".

Don't let the results stop you!

[LISNews]

From the article:
"Kelly Shannon, spokeswoman for the State Department's consular affairs office, said the Logan Airport results would not affect plans to use face recognition to enhance passport security"

So it doesn't work, won't help, and might even end up hurting more that a few people, but it's going to enhance passport security?

And Apparently OZ thinks it's a good idea too? "We now have an international standard established, which is the adoption of facial recognition as the international biometric, and that has left us well placed to move to implementation."

Re:Don't let the results stop you!

[RevMike]

So it doesn't work, won't help, and might even end up hurting more that a few people, but it's going to enhance passport security?

The article pointed out that the software was very effective at validating things like passport photos. One would imagine that a traveller would step up to the desk at customs/immigration and had over his passport. The immigration agent would insert the passport into a scanner. A camera would shoot a similar shot of the person standing at the counter. The software would then compare the two images and determine with a fairly high degree of reliability that the person at the counter is or is not the person in the passport photo. This determination could occur regardless of whether the person had gained or lost weight, lost hair, dyed hair, grew facial hair or shaved, or simply aged.

People make mistakes in this situation all the time. there is nothing wrong with having a computer try it.

Re:Don't let the results stop you!

[BrynM]

There's a few problems with the "scan a passport and compare" method. First, both forms of identification would be supplied by the person being screened (their face and their ID/Passport), which leaves wiggle room for tampering.

Second, some TSA lackey is going to get in the habit of passing IDs and passports under a scanner and looking for a result. They will think even less about comparing the face with the image for themselves. They will simply trust the computer. There's a great TSA article at Wired (Confessions of a Baggage Screener - Wired 11.09) that lays a lot of their habits bare.

Lastly, as someone already mentioned, the 9/11 attackers used their real names and real passports. Just because we are looking for terrorists, it doesn't mean we actually know who they are or what they look like.

I don't think that face recognition will help much but some department's budget and some politician's "knee-jerk" contribution efforts. Ok, this may prevent Osama from flying, but I don't think he'd assign himself to a suicide mission. It will always be some "volunteer" that we have very little record of.

Wouldn't the false positive rate be more important

[tbase]

Seems like the flase positive rate would be the most important stat, and they don't have it.

Obviously it couldn't replace ANY other security measure, but if it worked 61% of the time with NO false positives, I would call that pretty damn successful, especially in such an early implementation.

They said 10 of the 19 hijackers went through Logan - so this system theorhetically would have caught 6 of them? Better than none. And it seems like the technology would improve with time.

Personally I'd rather have my face scanned then have them strip searching me because my credit sucks and I paid cash for my plane ticket.

Of course it doesn't work.

[grub]

They aren't at the stage yet where machines can recognize people based on gait and mannerism. Facial recognition is a best guess and still requires a human to be sure of the fact just like fingerprint systems.

Re:Of course it doesn't work.

[cant_get_a_good_nick]

They aren't at the stage yet where machines can recognize people based on gait ....

So the Minister of Funny Walks is still safe.

I think the concept is being distorted

[TekReggard]

I think the concept of facial recognition is being distorted from a tool to help assist in the confirmation of possible criminals and terrorists to a single device that does all the work. The idea that it can still bring in a 61% accuracy rate is pretty good if you compare it to previous technologies. When you combine that with on the ground security and other systems, it only makes it easier for the Airport security, or other government or commerce locations to keep a tight hold on who comes in and out of their systems. Think of it this way, They have facial recognition working at entrances, and a few places along the way to security checkpoints. It picks up 3 positives out of 5000 people for terrorists or criminals (This is just a guess). They send that information to security at checkpoints with a picture from the camera, and whatever might be in a database. The security will be able to check to see if it was accurate at the checkpoint, and make a decision based on that for whether or not to check this person more throughly, stop them, arrest them, etc... whatever the case may be. If they can tell from the photograph that it was probably a false positive, they can just avoid it all together, or pick them for a random security check. No one said if it comes up with a false positive that person will be automatically picked up and thrown in jail. It sounds like a reasonable tool to help identify people they need to check more closely, nothing more.

Same old song and dance.... Snake oil sir?

[B5_geek]

As with most biometric systems, this is only ever works reliably in a lab.

Remeber the fingerprint system that got fooled by gelatine-gummi's ?

I wonder when these dot-bomb ideas will stop popping back up, and more credible research will get the much needed funds.

There is only one thing that has ever been able to recognize the human face; other humans. (And we do a rather poor job of it too after 10 million years of evolution!!!)

Proof: Take your average ignorant North American, (like myself) and ask him to tell the difference between 3 different Asian individuals. There is a good chance that we would fail that test because we are not used to (or mentally trained to) spot the difference.

{I love using myself for proof, it's so scientific}

Once again...

[CGP314]

A government test in 2002 found that face-recognition systems scored correct matches more than 90% of the time when used for such one-to-one identifications.

Once again, the false positives are not given. That is the number that really matters in a society where you can be held in prison indefinitely without a trial or access to a lawyer.

Typical results when a product is misused

[mnmlst]

Oddly, I read the article (unusual for a Slashdotter) here and it seems to imply that these companies were marketing their products for the limited use of trying to catch people with forged identity documents. Rather than just having a Customs officer compare the photograph to the face next to that photograph, the software could chime in with "Yeah, that's her alright." It looks like the security people at Logan Airport deployed these products in bulk. I wonder whose bright idea it was to try and use these at randumb? Perhaps a zealous salesperson or an overenthusiastic security manager? I also noticed the company spokesperson sounded a bit "hedged" like the company is trying to state that "Gosh, this product was never meant to be USED the way this customer is using it." The part left unsaid by the spokesperson was, "We told them this wouldn't work..." On a side note, let's not even consider how abysmal this software must perform when terrorists are deliberately disguising their faces.

Re:Typical results when a product is misused

[MightyTribble]

Yes, indeed. Visage is very fond of saying their system is designed for 1:1 comparisions, not database searches, and that it has a 90% success rate.

There are 2 problems with this, though:

The first is the false-positive rate. Visage is saying that, nine times out of ten, they can tell if the person being presented for inspection matches the photo. But what if they incorrectly flag one out of every fifteen users as *not* matching the picture? More work for Border control, that's what. The Mark One Eyeball is still the fastest, cheapest, best tool for comparing photos to people.

Second, it pays no mind to *false* papers with *correct* photographs. Sure, their fancy system will say "Yup, the person pictured is standing in front of you!" but if the underlying documentation is fake, so what?

Visage is a private company chasing lucrative federal dollars. All they need to do is create a product good enough to persuade Federal agencies to buy it - they don't actually need to make sure it does anything useful.

Re:Wouldn't the false positive rate be more import

[JohnGrahamCumming]

They said 10 of the 19 hijackers went through Logan - so this system theorhetically would have caught 6 of them? Better than none

The 9/11 hijackers used their real names and real ID. If they'd been placed on a simple watch list of names then strcmp would have found them, not some highfalutin' face recognition system. It's not the technology here, but coordination between the three letter agencies that's needed.

John.

Where's the false positive data?

[edwilli]

I'm not sure it's even worth looking at the data if we can't have some idea of the level of false positives. If you can find 1 out of 1000 criminals that walk by seems that it might be worth it?

What you don't want is harassing innocent people. If we can aviod that, I don't see where the problem is.

Have you kicked your kitten lately?

Why was it even deployed?

[Mrs. Grundy]

I wonder why the inaccuracy of this system wasn't well known before it was put up in a public place. Did it perform much better under the controlled environment of the lab? The article states that it works well in a one-to-one test, but they knew that this isn't how how it would be used in this case. It seems likely that if this failed so miserably in real life it couldn't have been that great when they were developing it. Does this speak of a certain desperation on the part of law enforcement to 'do something' or at least to appear to be doing something. Or maybe a hopefullness on the part of the company developing it that they might just get lucky. In fact, if they were payed by the government to deploy this test even though it seems likely they knew it would fail, maybe they did get lucky. Who payed for all this anyway?

Re:What is failure?

[fingon]

Well, assuming 100 percent false positive rate is okay too, I can sell you box that detects 100 percents of terrorists! And cheaper than what the govt pays for it's system, I bet! (just 100M $ for you, my friend..)

My disguise

[Mighty LoPan]

Okay, I admit it. I single-handedly foiled big brother's plan by marching around Logan with novelty glasses and a giant foam cowboy hat.

Re:Wouldn't the false positive rate be more import

[timeOday]

Seems like the flase positive rate would be the most important stat, and they don't have it.

Oh, they have it. If they're avoiding mentioning it, assume the worst.

Re:Wouldn't the false positive rate be more import

[jspoon]

"They said 10 of the 19 hijackers went through Logan - sothis system theorhetically would have caught 6 of them? Better than none. And it seems like the technology would improve with time."

Not quite. It would only have caught those of the hijackers who were on the do-not-fly list, which we all know is a resounding success. Since the hijackers did nothing to arouse suspicion in their initial period in the US, it's unlikely they'd have been flagged.

Engineers == naturally stoned

[J4]

Who comes up with harebrained schemes like this anyway?
It sounds like something a couple of potheads thought up.

Engr1: Dude, you know what'd be awesome? We could make a widget that recognizes faces, then we could put it at the door so we'd know if it was the pizza guy knocking.

Engr1: Whoaaa dude, that'd be awesome. Pass the caffeine.

Re:Face recognition

[jedidiah]

The current state of the art in facial recognition doesn't allow for automated operation. Anyone familiar with the technology realizes this. Articles such as this are no great revelation to anyone with a clue.

The current tech certainly is imperfect. However, it does offer the potential to allow increased scrutiny of a subset of passengers. This alone has value. We just have to decide how many false positives we can manage.

Just contemplate if 5 of the 10 hijackers at Logan had been detected. Every little bit can help.

Re:Geez, you people could at least read the headli

[Skuli]

If a false positive indicates someone who was incorrectly identified as a positive match, then a false negative would surely mean someone who was incorrectly identified as a negative match. Brian K

Twins?

[jhughes]

I haven't followed facial recognition too much but...wouldn't twins have an issue with this?
I'm an identical twin, I've had my lights punched out by someone who thought I was him (Thanks bro..grumble)...

Anyone able to tell me how this would differentiate between siblings that look very very much alike?

Re:Twins?

[Pvt_Waldo]

I bet it was at that point where you said, "It was my twin brother, really!" when the guy decided to take a swing at you :^)

Re:Wouldn't the false positive rate be more import

[awch]

The ACLU web site has their release.

It quotes the Logan report saying, "the number of system-generated false positives was excessive."

Failure depends on false positives

[riptalon]

The most important characteristic of such a system is the false positive rate. A system that flags everyone who passes through will flag 100 percent of terrorists but would be no better than having no system at all. They do not give the false positive rate but it is highly unlikely to be less than 10 percent and may be much larger. Since the ratio of terrorists to non-terrorists is probably on the order of a billion to one a system with an unrealistically low 1 percent false positive rate will flag 10 million non-terrorists for every 0.5 terrorists if it has a 50 percent correct ID rate. Even if you do extra searches on those 10 million people, with a 50 percent correct ID rate the terrorist is just a likely to be in the 990 million people who do not get flagged as in the flagged group.

You need a close to 100 percent correct ID rate and a false positive rate below one in a million, which is probably impossible, before the system would be of any use. However all this assumes that you have pictures of all terrorists. This is just plain impossible, especially in the case of suicide attacks. This is not like bank robbers where there are multiple incidents allowing evidense from witnesses etc. to be used to catch them when they try again. With suicide attack the attackers will likely be model citizens (who will not be on any list) right up until the attack and afterwards any info on them that is gathered is close to useless.

Face-Recognition System & Visa Application to

[reporter]

The tests conducted thus far on the face-recognition system shows that it cannot identify a particular face within a crowd of faces. However, this failure does not mean that the system has no useful application.

The system can be used to recognize a particular face when it is standing alone. Consider, for example, a photo of a face sent along with an visa application to the American embassy. Please read "World: Asia-Pacific China backs embassy protests". In 1999, Serbians committed gross human-rights violations against the Kosovars in Kosovo; the Chinese fully supported the Serbians in their campaign of terror. The North Atlantic Treaty Organization (NATO) under American command attempted to stop the slaughter by knocking out Serbian military units. NATO deliberately attempted to avoid hitting civilian targets in Serbia, but some bombs accidentally hit the Chinese embassy.

Shortly thereafter, the Chinese in both China and outside China erupted into ugly, violent protests. The Chinese throw stones and other projectiles at the American embassy in China. The Chinese also attacked some Americans. " The residence of the US Consul General in the south-western city of Chengdu was stormed and partially burned ."

How could the Americans in China have responded to this nonsensical violence? The Americans should have done the following.

1. Pull out cameras and take pictures of all the protestors.
2. Scan the photos into a computer and transmit them to Washington.
3. Henceforth, when a Chinese submits an application for a visa to travel to the USA, use the face-recognition system to determine whether the photo of the applicant matches any of the protestors. If there is a match, then the application will not be approved.
4. At the American embassy, grab a megaphone and loudly announce, "Attention protesters. We are using a face-recognition system. Any protestor applying for a visa to the USA will be denied entry into the USA. We are taking pictures right now. "

After about 10 clicks of the shutter of the camera, all the protestors would have disappeared. Henceforce, we should use this face-recognition system in conjunction with photographic equipment at all embassies and consulates run by Western nations within China (which includes Taiwan and Hong Kong).

... from the desk of the reporter

Can't we focus on something besides their accuracy

[Cyberllama]

I dislike the notion of being watched, categorized, and monitored everywhere I go. At this point, facial recognition systems have proven to be relatively inaccurate, and thus they have failed to gain widespread acceptance.

Proposals for facial recognition systems continue to be shot down because of their inaccuracy, but why does it have to be their inaccuracy that is the sticky point. Shouldn't the fact that they constitute a massive invasion of privacy be all the argument we need?

If we continue to use the "accuracy" argument over and over, then what happens when a system that is proven to be fully accurate comes out?

Facial Recognition Systems aren't a bad idea becuase they're inaccurate, they're simply a bad idea -- and that is what we should focus on.

Re:Wouldn't the false positive rate be more import

[jchristo]

I rather suspect that the false positive rate would make this system impractical. If there were only one false identification per ten thousand people, then the false positives would significantly outnumber the true identifications. This has been pointed out to be the real weakness in biometric identification systems.

Too Many False Positives

[Rick Richardson]

Taken from the ACLU web site:

According to the Logan report, which was written by an independent security contractor, "the number of system-generated false positives was excessive, and as a result, the operator's workload is taxing and strenuous, requiring constant undivided attention and periodic relief, which amounts to a staffing minimum of two persons for one workstation."

we need quantum computers!

[donkiemaster]

With quantum computer problems like this will be a thing of the past. We will just say "locate all criminals" and then the quantum computer will do it because they know things without actually knowing them. We will tell them to find the crimincals but they will have already found them, before the criminals even knew they were coming. Quantum computers will also be good for telling us what TV to watch and then rather than just watching the TV it will tell us whether the show we were going to watch was any good. Saves time that way. And then they never really had to make the show because the quantum computer already knew if we were going to like it so there is no real point in actually spending money to make the show. At least, this is my understanding of how quantum computers work. I think you can also do floating point math up to 10 digits too, but I wouldn't hold my breath.

61%? I know why it didn't work...

[r_glen]

The problem is NOT that the technology isn't there yet. Sloppy coding is to blame.
I've taken the libery of fixing it. Now we can finally replace those hard-working security screeners:

BOOL IsLikelyTerrorist (PASSENGER_INFO *passenger)
{
if (( passenger->skin_color != RGB(255,255,255) ) && ( strlen(passenger->name) > 30 ))
return TRUE;
else
return FALSE;
}

No thanks needed; I'm just glad I could help my country.