Slashdot Mirror
Facial Recognition Fails in Boston, Too
bryan writes "Only a few weeks after cameras were found to be ineffective in catching criminals in Tampa, FL, a test of a facial-recognition system in Boston's Logan airport also came up disappointing. The cameras which were given photos of employees to detect, were only successful in 153 out of 249 random tests over the past year (about 61%). The article did not say how many false positives the tests generated. The companies involved were Indentix and Visage."
As super-duper as high-tech is, I think even /.'ers would admit that its not a panacea (yet) for all our security ills. The very idea of having a computer capable of accurately identifying one face in thousands -- scanning from afar -- is far fetched. Despite billions in research we've yet to master voice recognition which is, comparatively, much easier to do.
Ah well, what's another few hundred million of tax payer's money shot. I'm sure it made some contractor rich.
CommentBot 0.7a running with args "-module irritate,disagree -target random"
"Kelly Shannon, spokeswoman for the State Department's consular affairs office, said the Logan Airport results would not affect plans to use face recognition to enhance passport security"
So it doesn't work, won't help, and might even end up hurting more that a few people, but it's going to enhance passport security?
And Apparently OZ thinks it's a good idea too? "We now have an international standard established, which is the adoption of facial recognition as the international biometric, and that has left us well placed to move to implementation."
Seems like the flase positive rate would be the most important stat, and they don't have it.
Obviously it couldn't replace ANY other security measure, but if it worked 61% of the time with NO false positives, I would call that pretty damn successful, especially in such an early implementation.
They said 10 of the 19 hijackers went through Logan - so this system theorhetically would have caught 6 of them? Better than none. And it seems like the technology would improve with time.
Personally I'd rather have my face scanned then have them strip searching me because my credit sucks and I paid cash for my plane ticket.
666-607: 6th floor apartment of the beast
Anything about whether employees attempted to disguise their appearance at all. If not I would hate to see the rate if they did.
Either way I don't I like it.
As with most biometric systems, this is only ever works reliably in a lab.
Remeber the fingerprint system that got fooled by gelatine-gummi's ?
I wonder when these dot-bomb ideas will stop popping back up, and more credible research will get the much needed funds.
There is only one thing that has ever been able to recognize the human face; other humans. (And we do a rather poor job of it too after 10 million years of evolution!!!)
Proof: Take your average ignorant North American, (like myself) and ask him to tell the difference between 3 different Asian individuals. There is a good chance that we would fail that test because we are not used to (or mentally trained to) spot the difference.
{I love using myself for proof, it's so scientific}
"The price good men pay for indifference to public affairs is to be ruled by evil men." ~Plato (427-347 BC)
A government test in 2002 found that face-recognition systems scored correct matches more than 90% of the time when used for such one-to-one identifications.
Once again, the false positives are not given. That is the number that really matters in a society where you can be held in prison indefinitely without a trial or access to a lawyer.
They said 10 of the 19 hijackers went through Logan - so this system theorhetically would have caught 6 of them? Better than none
The 9/11 hijackers used their real names and real ID. If they'd been placed on a simple watch list of names then strcmp would have found them, not some highfalutin' face recognition system. It's not the technology here, but coordination between the three letter agencies that's needed.
John.
I'm not sure it's even worth looking at the data if we can't have some idea of the level of false positives. If you can find 1 out of 1000 criminals that walk by seems that it might be worth it?
What you don't want is harassing innocent people. If we can aviod that, I don't see where the problem is.
Have you kicked your kitten lately?
I wonder why the inaccuracy of this system wasn't well known before it was put up in a public place. Did it perform much better under the controlled environment of the lab? The article states that it works well in a one-to-one test, but they knew that this isn't how how it would be used in this case. It seems likely that if this failed so miserably in real life it couldn't have been that great when they were developing it. Does this speak of a certain desperation on the part of law enforcement to 'do something' or at least to appear to be doing something. Or maybe a hopefullness on the part of the company developing it that they might just get lucky. In fact, if they were payed by the government to deploy this test even though it seems likely they knew it would fail, maybe they did get lucky. Who payed for all this anyway?
If and when they happen, it's a simple matter to clear up a person's real identity. It's not like they shoot first and ask questions later.
Not quite, but they can hold you indefinitely, without access to a lawyer, without notifying your family, without even charging you. Better, but not by much. Thats OK, because terrorists are ++ungood and we'll trample any rights needed to preserve freedom.
But as others have mentioned the issue of false positives is also critical, just like in anti-spam software. If there are so many false psoitives that airport security is running around checking every other businessman then the system is nearly useless.
Yes, indeed. Visage is very fond of saying their system is designed for 1:1 comparisions, not database searches, and that it has a 90% success rate.
There are 2 problems with this, though:
The first is the false-positive rate. Visage is saying that, nine times out of ten, they can tell if the person being presented for inspection matches the photo. But what if they incorrectly flag one out of every fifteen users as *not* matching the picture? More work for Border control, that's what. The Mark One Eyeball is still the fastest, cheapest, best tool for comparing photos to people.
Second, it pays no mind to *false* papers with *correct* photographs. Sure, their fancy system will say "Yup, the person pictured is standing in front of you!" but if the underlying documentation is fake, so what?
Visage is a private company chasing lucrative federal dollars. All they need to do is create a product good enough to persuade Federal agencies to buy it - they don't actually need to make sure it does anything useful.
They'd have been cleared and let on the flight, dumbass. They used thier real names and IDs. This technology wouldn't have prevented the Sept. 11th attacks, but may, with another 5 years or so of improvement, be of use in preventing other attacks with different methodology.
Just thought I'd chime in here. Technology is neutral. It can be used for good or ill. Facial recognition technology can be a great thing, if used properly for constructive purposes. For example, it could be used to help with identity recognition at ATM-machines.
Yes, these technologies are failing alot. But, just a couple of years ago, people would have scoffed at the idea that computers could even begin to accomplish some kind of face-recognition. This technology is in it's infant stages. I don't think you can blame a technology that's just gotten off the ground for not being perfect.
Lets criticize improper uses of this technology, not the technology itself.
social sciences can never use experience to verify their statemen
The most important characteristic of such a system is the false positive rate. A system that flags everyone who passes through will flag 100 percent of terrorists but would be no better than having no system at all. They do not give the false positive rate but it is highly unlikely to be less than 10 percent and may be much larger. Since the ratio of terrorists to non-terrorists is probably on the order of a billion to one a system with an unrealistically low 1 percent false positive rate will flag 10 million non-terrorists for every 0.5 terrorists if it has a 50 percent correct ID rate. Even if you do extra searches on those 10 million people, with a 50 percent correct ID rate the terrorist is just a likely to be in the 990 million people who do not get flagged as in the flagged group.
You need a close to 100 percent correct ID rate and a false positive rate below one in a million, which is probably impossible, before the system would be of any use. However all this assumes that you have pictures of all terrorists. This is just plain impossible, especially in the case of suicide attacks. This is not like bank robbers where there are multiple incidents allowing evidense from witnesses etc. to be used to catch them when they try again. With suicide attack the attackers will likely be model citizens (who will not be on any list) right up until the attack and afterwards any info on them that is gathered is close to useless.
I dislike the notion of being watched, categorized, and monitored everywhere I go. At this point, facial recognition systems have proven to be relatively inaccurate, and thus they have failed to gain widespread acceptance.
Proposals for facial recognition systems continue to be shot down because of their inaccuracy, but why does it have to be their inaccuracy that is the sticky point. Shouldn't the fact that they constitute a massive invasion of privacy be all the argument we need?
If we continue to use the "accuracy" argument over and over, then what happens when a system that is proven to be fully accurate comes out?
Facial Recognition Systems aren't a bad idea becuase they're inaccurate, they're simply a bad idea -- and that is what we should focus on.
You miss the point of the article. This was a small scale test where only about 140? individuals data was entered.
If you ramp this up to a full system then the following happens:
1. The biometric data of MILLIONS of people has to be stored. This exceeds current capacities of available computers.
2. Response time will SUCK worse than current x-ray machine lines.
3. The false positive and false negative rates will soar with the addition of the other data to compare to.
Lets face it, this test failed MISERABLY. It is time to take our tax dollars elsewhere.
The number of shaky premises, false inferences, and unjustified conclusions in your post, is right up there with mainstream journalism.
Rather than show all these problems in formal logic, let me just point out one problem:
A humans ability to do facial recognition has nothing to do with a computer's inability to do it.
Also let me throw in the technical point that any parallel solution to the problem, can be emulated serially; it just will take more time and probably more hardware to do it.
1) They aren't any more effective than human personal, and unless they have an unrealistically low false-postive rate, they actually generate more work than they're worth
2) As anyone who's been to (or worked in) a DMV knows, there's basically 2 human reponses to computerized systems like this: a) you ignore the computer whenever it disagrees with you or b) you always obey the computer, no matter what. Unless you've got fantastically high sucess rates, b is exactly what the tinfoil croud is worried about (because this is even more important than getting the right address on your drivers license). A means that the computer system is a waste of time and money, which is the most common result if the system has low accuracy.
I rather suspect that the false positive rate would make this system impractical. If there were only one false identification per ten thousand people, then the false positives would significantly outnumber the true identifications. This has been pointed out to be the real weakness in biometric identification systems.
My dog does pretty well.
I've found that my posts don't format quite right w/o a sig.