Facial Recognition Fails in Boston, Too

bryan writes "Only a few weeks after cameras were found to be ineffective in catching criminals in Tampa, FL, a test of a facial-recognition system in Boston's Logan airport also came up disappointing. The cameras which were given photos of employees to detect, were only successful in 153 out of 249 random tests over the past year (about 61%). The article did not say how many false positives the tests generated. The companies involved were Indentix and Visage."

Comments

The system would work better...

[MikeCapone]

...if the cameras were mounted on black helicopters.

wetware comparison

[Empiric]

I heard that a very substantial amount of our brain's capacity is devoted to differentiating faces, and it's conjectured that this processing overkill is responsible for such things as people seeing a "face" in the objectively very non-face-like features of the moon.

Give the parallel processing capability people have to do this trick, it's probably not too surprising that computer tech hasn't gotten there yet.

Anyone know more about face-recognition processing in the human brain? I find this topic quite interesting...

Re:wetware comparison

[bytesmythe]

I can add a little to this...

I wouldn't call our system "overkill". Also, there are really multiple systems involved.

First, there is the subsystem that recognizes a face as being a face. There are certain clusters of neurons that fire in response to any face-like pattern, regardless of whether or not it is actually attached to a head. This is how we recognize animal faces as being faces, the man in the moon, smiley faces and emoticons, Jesus in a water stain, etc. This capacity is innate, and infants can discern face-like patterns very soon after being born.

After a face has been perceived, it must be narrowed down to an individual person. This ability is partly learned over time, and is responsible for the difficulty people have in recognizing faces outside of their own cultural group. Certain types of brain damage (from a stroke, for instance) can allow people to recognize the fact that they're looking at a face, but still be unable to determine whose face they're looking at.

Keep in mind that even before a face is perceived, you have systems that find the basic shape outlines, determine their orientations (separately and with respect to each other), and at the same time attach color information, shuttle it off to "face subprocessing", then call up any related emotional context (have you ever seen a stranger you didn't like because they resemble someone you already dislike?) -- all before you can become consciously aware of the face.

Re:wetware comparison

[wirelessbuzzers]

Caveat: following is IIRC; this is from a basic developmental psych class that I took for breadth requirements. I got a lousy grade in the class mostly for failing to remember specific details, so some of this may be just plain wrong. Go to Scirus or something and search for this stuff before using it in your doctoral thesis.

Human face recognition is also built-in. Psych experiments on newborns (straight out of the maternity ward), as well as older infants, indicate that they can detect faces early on. This is not the case with all visual abilities that people have! There are many types of spacial recognition and object-parsing tasks that infants, and even toddlers, simply can't do.

Newborns pay more attention to shapes that look like a face > over those that are schematically similar > over those those with eyes, noses etc but in different arrangements > over ovals with random junk > over blank ovals...

Also at an early age (don't remember when; I don't think they tested it on newborns, but I wouldn't be surprised if they can do this too), babies can tell familiar faces from unfamiliar ones, and show an inverted habituation effect; that is, they prefer to look at familiar faces than unfamiliar ones (unlike most shapes, where they get bored with the familiar ones).

Also at an early age (again, I don't remember how early, but less than a year), infants map others' faces onto their own and imitate. That is, if you stand in front of them for a few minutes with your tongue sticking out, they often stick theirs out too. If you have one eye open and one closed, they'll copy that too (I don't remember which eye tho).

From an early age, babies can also follow gazes to tell what someone is looking at; this is important in the development of language as well as vision, because babies use it to figure out what an adult is talking about. It is, IIRC, used more than what the adult is *pointing* at.

In addition to recognizing faces, babies can recognise other body parts, and treat an action differently based on whether it is done by a hand or a stick (when they don't see the hand holding the stick). If it is done by a hand or other object perceived to be animate, it is treated as goal-oriented and categorized partly by the perceived goal; if done by an "inanimate" object, the baby does not look for a goal. This is studied through habituation experiments; different actions with the same goal were seen as more similar (and hence less interesting) than those with the same basic appearance, if and only if they were performed by an animate-looking object such as a hand.

Re:wetware comparison

[greenhide]

Not susceptible to any sort of bias?!?

May I direct you to the following quote, from a highly notable artificial intelligence program:

I hate this place, this zoo, this prison, this reality, whatever you want to call it. I can't stand it anymore. It's the smell, if there is such a thing. I can taste your stink, and every time I do, I fear I've somehow been contaminated by it.

Proof!

Computers hate us to their very bones. My computer has only crashed when I've been doing something important, like writing a term paper or surfing for porn. They're out to get us, all right.

Improbable to start with

[lawaetf1]

As super-duper as high-tech is, I think even /.'ers would admit that its not a panacea (yet) for all our security ills. The very idea of having a computer capable of accurately identifying one face in thousands -- scanning from afar -- is far fetched. Despite billions in research we've yet to master voice recognition which is, comparatively, much easier to do. Ah well, what's another few hundred million of tax payer's money shot. I'm sure it made some contractor rich.

Re:Improbable to start with

[krymsin01]

Yeah, but you have a government who is willing to spen the taxpayers money on this sort of thing. By and large, most taxpayers do not care about their privacy being taken away from them under the guise of security. Even if they did, you would think that more than the less than half of the population that votes would actualy vote to stop it.

As for it not being there yet, a lot of people said it was a far fetched idea for the US to send people to the moon, and in fact, a few people still believe that it didn't happen and it couldn't have. I'm willing to accept that it did happen, because the US Government wanted to show up the Russians and beat them to it. They were willing to spend the money, the technology emerges. Same thing here. If the government wants the tech, all they have to do is throw money at it, and wait. It'll eventualy be here before you know it.

Point of view

[mattdm]

I'm not sure I would call the failure of big-brother tech "disappointing".

Don't let the results stop you!

[LISNews]

From the article:
"Kelly Shannon, spokeswoman for the State Department's consular affairs office, said the Logan Airport results would not affect plans to use face recognition to enhance passport security"

So it doesn't work, won't help, and might even end up hurting more that a few people, but it's going to enhance passport security?

And Apparently OZ thinks it's a good idea too? "We now have an international standard established, which is the adoption of facial recognition as the international biometric, and that has left us well placed to move to implementation."

Re:Don't let the results stop you!

[RevMike]

So it doesn't work, won't help, and might even end up hurting more that a few people, but it's going to enhance passport security?

The article pointed out that the software was very effective at validating things like passport photos. One would imagine that a traveller would step up to the desk at customs/immigration and had over his passport. The immigration agent would insert the passport into a scanner. A camera would shoot a similar shot of the person standing at the counter. The software would then compare the two images and determine with a fairly high degree of reliability that the person at the counter is or is not the person in the passport photo. This determination could occur regardless of whether the person had gained or lost weight, lost hair, dyed hair, grew facial hair or shaved, or simply aged.

People make mistakes in this situation all the time. there is nothing wrong with having a computer try it.

Re:Don't let the results stop you!

[BrynM]

There's a few problems with the "scan a passport and compare" method. First, both forms of identification would be supplied by the person being screened (their face and their ID/Passport), which leaves wiggle room for tampering.

Second, some TSA lackey is going to get in the habit of passing IDs and passports under a scanner and looking for a result. They will think even less about comparing the face with the image for themselves. They will simply trust the computer. There's a great TSA article at Wired (Confessions of a Baggage Screener - Wired 11.09) that lays a lot of their habits bare.

Lastly, as someone already mentioned, the 9/11 attackers used their real names and real passports. Just because we are looking for terrorists, it doesn't mean we actually know who they are or what they look like.

I don't think that face recognition will help much but some department's budget and some politician's "knee-jerk" contribution efforts. Ok, this may prevent Osama from flying, but I don't think he'd assign himself to a suicide mission. It will always be some "volunteer" that we have very little record of.

Wouldn't the false positive rate be more important

[tbase]

Seems like the flase positive rate would be the most important stat, and they don't have it.

Obviously it couldn't replace ANY other security measure, but if it worked 61% of the time with NO false positives, I would call that pretty damn successful, especially in such an early implementation.

They said 10 of the 19 hijackers went through Logan - so this system theorhetically would have caught 6 of them? Better than none. And it seems like the technology would improve with time.

Personally I'd rather have my face scanned then have them strip searching me because my credit sucks and I paid cash for my plane ticket.

Of course it doesn't work.

[grub]

They aren't at the stage yet where machines can recognize people based on gait and mannerism. Facial recognition is a best guess and still requires a human to be sure of the fact just like fingerprint systems.

Re:Of course it doesn't work.

[cant_get_a_good_nick]

They aren't at the stage yet where machines can recognize people based on gait ....

So the Minister of Funny Walks is still safe.

I think the concept is being distorted

[TekReggard]

I think the concept of facial recognition is being distorted from a tool to help assist in the confirmation of possible criminals and terrorists to a single device that does all the work. The idea that it can still bring in a 61% accuracy rate is pretty good if you compare it to previous technologies. When you combine that with on the ground security and other systems, it only makes it easier for the Airport security, or other government or commerce locations to keep a tight hold on who comes in and out of their systems. Think of it this way, They have facial recognition working at entrances, and a few places along the way to security checkpoints. It picks up 3 positives out of 5000 people for terrorists or criminals (This is just a guess). They send that information to security at checkpoints with a picture from the camera, and whatever might be in a database. The security will be able to check to see if it was accurate at the checkpoint, and make a decision based on that for whether or not to check this person more throughly, stop them, arrest them, etc... whatever the case may be. If they can tell from the photograph that it was probably a false positive, they can just avoid it all together, or pick them for a random security check. No one said if it comes up with a false positive that person will be automatically picked up and thrown in jail. It sounds like a reasonable tool to help identify people they need to check more closely, nothing more.

Same old song and dance.... Snake oil sir?

[B5_geek]

As with most biometric systems, this is only ever works reliably in a lab.

Remeber the fingerprint system that got fooled by gelatine-gummi's ?

I wonder when these dot-bomb ideas will stop popping back up, and more credible research will get the much needed funds.

There is only one thing that has ever been able to recognize the human face; other humans. (And we do a rather poor job of it too after 10 million years of evolution!!!)

Proof: Take your average ignorant North American, (like myself) and ask him to tell the difference between 3 different Asian individuals. There is a good chance that we would fail that test because we are not used to (or mentally trained to) spot the difference.

{I love using myself for proof, it's so scientific}

Once again...

[CGP314]

A government test in 2002 found that face-recognition systems scored correct matches more than 90% of the time when used for such one-to-one identifications.

Once again, the false positives are not given. That is the number that really matters in a society where you can be held in prison indefinitely without a trial or access to a lawyer.

Typical results when a product is misused

[mnmlst]

Oddly, I read the article (unusual for a Slashdotter) here and it seems to imply that these companies were marketing their products for the limited use of trying to catch people with forged identity documents. Rather than just having a Customs officer compare the photograph to the face next to that photograph, the software could chime in with "Yeah, that's her alright." It looks like the security people at Logan Airport deployed these products in bulk. I wonder whose bright idea it was to try and use these at randumb? Perhaps a zealous salesperson or an overenthusiastic security manager? I also noticed the company spokesperson sounded a bit "hedged" like the company is trying to state that "Gosh, this product was never meant to be USED the way this customer is using it." The part left unsaid by the spokesperson was, "We told them this wouldn't work..." On a side note, let's not even consider how abysmal this software must perform when terrorists are deliberately disguising their faces.

Re:Typical results when a product is misused

[MightyTribble]

Yes, indeed. Visage is very fond of saying their system is designed for 1:1 comparisions, not database searches, and that it has a 90% success rate.

There are 2 problems with this, though:

The first is the false-positive rate. Visage is saying that, nine times out of ten, they can tell if the person being presented for inspection matches the photo. But what if they incorrectly flag one out of every fifteen users as *not* matching the picture? More work for Border control, that's what. The Mark One Eyeball is still the fastest, cheapest, best tool for comparing photos to people.

Second, it pays no mind to *false* papers with *correct* photographs. Sure, their fancy system will say "Yup, the person pictured is standing in front of you!" but if the underlying documentation is fake, so what?

Visage is a private company chasing lucrative federal dollars. All they need to do is create a product good enough to persuade Federal agencies to buy it - they don't actually need to make sure it does anything useful.

Re:Wouldn't the false positive rate be more import

[JohnGrahamCumming]

They said 10 of the 19 hijackers went through Logan - so this system theorhetically would have caught 6 of them? Better than none

The 9/11 hijackers used their real names and real ID. If they'd been placed on a simple watch list of names then strcmp would have found them, not some highfalutin' face recognition system. It's not the technology here, but coordination between the three letter agencies that's needed.

John.

My disguise

[Mighty LoPan]

Okay, I admit it. I single-handedly foiled big brother's plan by marching around Logan with novelty glasses and a giant foam cowboy hat.

Re:Wouldn't the false positive rate be more import

[timeOday]

Seems like the flase positive rate would be the most important stat, and they don't have it.

Oh, they have it. If they're avoiding mentioning it, assume the worst.

Engineers == naturally stoned

[J4]

Who comes up with harebrained schemes like this anyway?
It sounds like something a couple of potheads thought up.

Engr1: Dude, you know what'd be awesome? We could make a widget that recognizes faces, then we could put it at the door so we'd know if it was the pizza guy knocking.

Engr1: Whoaaa dude, that'd be awesome. Pass the caffeine.

Twins?

[jhughes]

I haven't followed facial recognition too much but...wouldn't twins have an issue with this?
I'm an identical twin, I've had my lights punched out by someone who thought I was him (Thanks bro..grumble)...

Anyone able to tell me how this would differentiate between siblings that look very very much alike?

Re:Wouldn't the false positive rate be more import

[awch]

The ACLU web site has their release.

It quotes the Logan report saying, "the number of system-generated false positives was excessive."

Too Many False Positives

[Rick Richardson]

Taken from the ACLU web site:

According to the Logan report, which was written by an independent security contractor, "the number of system-generated false positives was excessive, and as a result, the operator's workload is taxing and strenuous, requiring constant undivided attention and periodic relief, which amounts to a staffing minimum of two persons for one workstation."

we need quantum computers!

[donkiemaster]

With quantum computer problems like this will be a thing of the past. We will just say "locate all criminals" and then the quantum computer will do it because they know things without actually knowing them. We will tell them to find the crimincals but they will have already found them, before the criminals even knew they were coming. Quantum computers will also be good for telling us what TV to watch and then rather than just watching the TV it will tell us whether the show we were going to watch was any good. Saves time that way. And then they never really had to make the show because the quantum computer already knew if we were going to like it so there is no real point in actually spending money to make the show. At least, this is my understanding of how quantum computers work. I think you can also do floating point math up to 10 digits too, but I wouldn't hold my breath.