Slashdot Mirror
Facial Recognition Fails in Boston, Too
bryan writes "Only a few weeks after cameras were found to be ineffective in catching criminals in Tampa, FL, a test of a facial-recognition system in Boston's Logan airport also came up disappointing. The cameras which were given photos of employees to detect, were only successful in 153 out of 249 random tests over the past year (about 61%). The article did not say how many false positives the tests generated. The companies involved were Indentix and Visage."
...if the cameras were mounted on black helicopters.
Treehugger? Treehugger... Treehugger!
It has 39% false negatives, not false positives!
Slashdotter are stupid and biased.
I think you have quite a misunderstanding of the difference between "false negative" (which is what the 61% was referring to) and "false positive" (which was not mentioned).
Send your friends messages of love at fuck-you.org
I heard that a very substantial amount of our brain's capacity is devoted to differentiating faces, and it's conjectured that this processing overkill is responsible for such things as people seeing a "face" in the objectively very non-face-like features of the moon.
Give the parallel processing capability people have to do this trick, it's probably not too surprising that computer tech hasn't gotten there yet.
Anyone know more about face-recognition processing in the human brain? I find this topic quite interesting...
~ Whence do you come, slayer of men, or where are you going, conqueror of space?
As super-duper as high-tech is, I think even /.'ers would admit that its not a panacea (yet) for all our security ills. The very idea of having a computer capable of accurately identifying one face in thousands -- scanning from afar -- is far fetched. Despite billions in research we've yet to master voice recognition which is, comparatively, much easier to do.
Ah well, what's another few hundred million of tax payer's money shot. I'm sure it made some contractor rich.
CommentBot 0.7a running with args "-module irritate,disagree -target random"
I'm not sure I would call the failure of big-brother tech "disappointing".
"Kelly Shannon, spokeswoman for the State Department's consular affairs office, said the Logan Airport results would not affect plans to use face recognition to enhance passport security"
So it doesn't work, won't help, and might even end up hurting more that a few people, but it's going to enhance passport security?
And Apparently OZ thinks it's a good idea too? "We now have an international standard established, which is the adoption of facial recognition as the international biometric, and that has left us well placed to move to implementation."
Seems like the flase positive rate would be the most important stat, and they don't have it.
Obviously it couldn't replace ANY other security measure, but if it worked 61% of the time with NO false positives, I would call that pretty damn successful, especially in such an early implementation.
They said 10 of the 19 hijackers went through Logan - so this system theorhetically would have caught 6 of them? Better than none. And it seems like the technology would improve with time.
Personally I'd rather have my face scanned then have them strip searching me because my credit sucks and I paid cash for my plane ticket.
666-607: 6th floor apartment of the beast
They aren't at the stage yet where machines can recognize people based on gait and mannerism. Facial recognition is a best guess and still requires a human to be sure of the fact just like fingerprint systems.
Trolling is a art,
As it happens, a friend of mine is working for a company that are in this field....they successfully implemented eye tracking, (dont tell me, there are lots of companies doing this, but not as well as these guys..). We discussed it the other day, and he told me that the face-recognitions algoritms are coming along..there is some huge stastically problems involved in this, the equipment is not the problem (they are using ordinary webcams) and some special light(ir-freq)...pretty cool stuff. Now, the ethical problems with this are MUCH more difficult.
- While I myself say there is naught, nor ought there be,
Nothing so exalted on the face of God's grey earth
As that Prince of Foods . . . The Muffin!- The Muffin Man
Comment removed based on user account deletion
Anything about whether employees attempted to disguise their appearance at all. If not I would hate to see the rate if they did.
Either way I don't I like it.
I think the concept of facial recognition is being distorted from a tool to help assist in the confirmation of possible criminals and terrorists to a single device that does all the work. The idea that it can still bring in a 61% accuracy rate is pretty good if you compare it to previous technologies. When you combine that with on the ground security and other systems, it only makes it easier for the Airport security, or other government or commerce locations to keep a tight hold on who comes in and out of their systems. Think of it this way, They have facial recognition working at entrances, and a few places along the way to security checkpoints. It picks up 3 positives out of 5000 people for terrorists or criminals (This is just a guess). They send that information to security at checkpoints with a picture from the camera, and whatever might be in a database. The security will be able to check to see if it was accurate at the checkpoint, and make a decision based on that for whether or not to check this person more throughly, stop them, arrest them, etc... whatever the case may be. If they can tell from the photograph that it was probably a false positive, they can just avoid it all together, or pick them for a random security check. No one said if it comes up with a false positive that person will be automatically picked up and thrown in jail. It sounds like a reasonable tool to help identify people they need to check more closely, nothing more.
As with most biometric systems, this is only ever works reliably in a lab.
Remeber the fingerprint system that got fooled by gelatine-gummi's ?
I wonder when these dot-bomb ideas will stop popping back up, and more credible research will get the much needed funds.
There is only one thing that has ever been able to recognize the human face; other humans. (And we do a rather poor job of it too after 10 million years of evolution!!!)
Proof: Take your average ignorant North American, (like myself) and ask him to tell the difference between 3 different Asian individuals. There is a good chance that we would fail that test because we are not used to (or mentally trained to) spot the difference.
{I love using myself for proof, it's so scientific}
"The price good men pay for indifference to public affairs is to be ruled by evil men." ~Plato (427-347 BC)
A government test in 2002 found that face-recognition systems scored correct matches more than 90% of the time when used for such one-to-one identifications.
Once again, the false positives are not given. That is the number that really matters in a society where you can be held in prison indefinitely without a trial or access to a lawyer.
Oddly, I read the article (unusual for a Slashdotter) here and it seems to imply that these companies were marketing their products for the limited use of trying to catch people with forged identity documents. Rather than just having a Customs officer compare the photograph to the face next to that photograph, the software could chime in with "Yeah, that's her alright." It looks like the security people at Logan Airport deployed these products in bulk. I wonder whose bright idea it was to try and use these at randumb? Perhaps a zealous salesperson or an overenthusiastic security manager? I also noticed the company spokesperson sounded a bit "hedged" like the company is trying to state that "Gosh, this product was never meant to be USED the way this customer is using it." The part left unsaid by the spokesperson was, "We told them this wouldn't work..." On a side note, let's not even consider how abysmal this software must perform when terrorists are deliberately disguising their faces.
In principio erat Verbum.
Sounds impressive, actually. If there are 4-5 hijackers, and each has a 61% chance of being noticed, then the odds are good that at least one will be and the plot will be foiled.
Also, what's the worry about false positives? If and when they happen, it's a simple matter to clear up a person's real identity. It's not like they shoot first and ask questions later.
They said 10 of the 19 hijackers went through Logan - so this system theorhetically would have caught 6 of them? Better than none
The 9/11 hijackers used their real names and real ID. If they'd been placed on a simple watch list of names then strcmp would have found them, not some highfalutin' face recognition system. It's not the technology here, but coordination between the three letter agencies that's needed.
John.
I'm not sure it's even worth looking at the data if we can't have some idea of the level of false positives. If you can find 1 out of 1000 criminals that walk by seems that it might be worth it?
What you don't want is harassing innocent people. If we can aviod that, I don't see where the problem is.
Have you kicked your kitten lately?
Oh dear...
First, a poster of someone else's face (facial recognition evasion).
Second, the goey fingerprint duplicator,
now this walk-by signature hacker on a PDA?
What would be next?
Hijacking IRIS pattern (simply stareing at the bathroom mirror)?
Stolen DNA pattern?
There is no solid defense against unrevokable but stolen biometric parameters.
I wonder why the inaccuracy of this system wasn't well known before it was put up in a public place. Did it perform much better under the controlled environment of the lab? The article states that it works well in a one-to-one test, but they knew that this isn't how how it would be used in this case. It seems likely that if this failed so miserably in real life it couldn't have been that great when they were developing it. Does this speak of a certain desperation on the part of law enforcement to 'do something' or at least to appear to be doing something. Or maybe a hopefullness on the part of the company developing it that they might just get lucky. In fact, if they were payed by the government to deploy this test even though it seems likely they knew it would fail, maybe they did get lucky. Who payed for all this anyway?
Well, assuming 100 percent false positive rate is okay too, I can sell you box that detects 100 percents of terrorists! And cheaper than what the govt pays for it's system, I bet! (just 100M $ for you, my friend..)
-- pending
Okay, I admit it. I single-handedly foiled big brother's plan by marching around Logan with novelty glasses and a giant foam cowboy hat.
Not quite. It would only have caught those of the hijackers who were on the do-not-fly list, which we all know is a resounding success. Since the hijackers did nothing to arouse suspicion in their initial period in the US, it's unlikely they'd have been flagged.
Who comes up with harebrained schemes like this anyway?
It sounds like something a couple of potheads thought up.
Engr1: Dude, you know what'd be awesome? We could make a widget that recognizes faces, then we could put it at the door so we'd know if it was the pizza guy knocking.
Engr1: Whoaaa dude, that'd be awesome. Pass the caffeine.
The current state of the art in facial recognition doesn't allow for automated operation. Anyone familiar with the technology realizes this. Articles such as this are no great revelation to anyone with a clue.
The current tech certainly is imperfect. However, it does offer the potential to allow increased scrutiny of a subset of passengers. This alone has value. We just have to decide how many false positives we can manage.
Just contemplate if 5 of the 10 hijackers at Logan had been detected. Every little bit can help.
A Pirate and a Puritan look the same on a balance sheet.
But as others have mentioned the issue of false positives is also critical, just like in anti-spam software. If there are so many false psoitives that airport security is running around checking every other businessman then the system is nearly useless.
If a false positive indicates someone who was incorrectly identified as a positive match, then a false negative would surely mean someone who was incorrectly identified as a negative match. Brian K
I haven't followed facial recognition too much but...wouldn't twins have an issue with this?
I'm an identical twin, I've had my lights punched out by someone who thought I was him (Thanks bro..grumble)...
Anyone able to tell me how this would differentiate between siblings that look very very much alike?
It quotes the Logan report saying, "the number of system-generated false positives was excessive."
They'd have been cleared and let on the flight, dumbass. They used thier real names and IDs. This technology wouldn't have prevented the Sept. 11th attacks, but may, with another 5 years or so of improvement, be of use in preventing other attacks with different methodology.
Just thought I'd chime in here. Technology is neutral. It can be used for good or ill. Facial recognition technology can be a great thing, if used properly for constructive purposes. For example, it could be used to help with identity recognition at ATM-machines.
Yes, these technologies are failing alot. But, just a couple of years ago, people would have scoffed at the idea that computers could even begin to accomplish some kind of face-recognition. This technology is in it's infant stages. I don't think you can blame a technology that's just gotten off the ground for not being perfect.
Lets criticize improper uses of this technology, not the technology itself.
social sciences can never use experience to verify their statemen
it means 31% of the employees weren't recognized as employees. It should have been a positive match, but returned a negative, falsely.
The most important characteristic of such a system is the false positive rate. A system that flags everyone who passes through will flag 100 percent of terrorists but would be no better than having no system at all. They do not give the false positive rate but it is highly unlikely to be less than 10 percent and may be much larger. Since the ratio of terrorists to non-terrorists is probably on the order of a billion to one a system with an unrealistically low 1 percent false positive rate will flag 10 million non-terrorists for every 0.5 terrorists if it has a 50 percent correct ID rate. Even if you do extra searches on those 10 million people, with a 50 percent correct ID rate the terrorist is just a likely to be in the 990 million people who do not get flagged as in the flagged group.
You need a close to 100 percent correct ID rate and a false positive rate below one in a million, which is probably impossible, before the system would be of any use. However all this assumes that you have pictures of all terrorists. This is just plain impossible, especially in the case of suicide attacks. This is not like bank robbers where there are multiple incidents allowing evidense from witnesses etc. to be used to catch them when they try again. With suicide attack the attackers will likely be model citizens (who will not be on any list) right up until the attack and afterwards any info on them that is gathered is close to useless.
The system can be used to recognize a particular face when it is standing alone. Consider, for example, a photo of a face sent along with an visa application to the American embassy. Please read "World: Asia-Pacific China backs embassy protests". In 1999, Serbians committed gross human-rights violations against the Kosovars in Kosovo; the Chinese fully supported the Serbians in their campaign of terror. The North Atlantic Treaty Organization (NATO) under American command attempted to stop the slaughter by knocking out Serbian military units. NATO deliberately attempted to avoid hitting civilian targets in Serbia, but some bombs accidentally hit the Chinese embassy.
Shortly thereafter, the Chinese in both China and outside China erupted into ugly, violent protests. The Chinese throw stones and other projectiles at the American embassy in China. The Chinese also attacked some Americans. " The residence of the US Consul General in the south-western city of Chengdu was stormed and partially burned ."
How could the Americans in China have responded to this nonsensical violence? The Americans should have done the following.
-
Pull out cameras and take pictures of all the protestors.
-
Scan the photos into a computer and transmit them to Washington.
-
Henceforth, when a Chinese submits an application for
a visa to travel to the USA, use the face-recognition system
to determine whether the photo of the applicant matches any of the
protestors. If there is a match, then the application will not be approved.
- At the American embassy, grab a megaphone
and loudly announce, "Attention protesters. We are using
a face-recognition system. Any protestor applying for a visa to
the USA will be denied entry into the USA. We are taking pictures
right now. "
After about 10 clicks of the shutter of the camera, all the protestors would have disappeared. Henceforce, we should use this face-recognition system in conjunction with photographic equipment at all embassies and consulates run by Western nations within China (which includes Taiwan and Hong Kong).Cunningly facist, I like it.
I dislike the notion of being watched, categorized, and monitored everywhere I go. At this point, facial recognition systems have proven to be relatively inaccurate, and thus they have failed to gain widespread acceptance.
Proposals for facial recognition systems continue to be shot down because of their inaccuracy, but why does it have to be their inaccuracy that is the sticky point. Shouldn't the fact that they constitute a massive invasion of privacy be all the argument we need?
If we continue to use the "accuracy" argument over and over, then what happens when a system that is proven to be fully accurate comes out?
Facial Recognition Systems aren't a bad idea becuase they're inaccurate, they're simply a bad idea -- and that is what we should focus on.
You miss the point of the article. This was a small scale test where only about 140? individuals data was entered.
If you ramp this up to a full system then the following happens:
1. The biometric data of MILLIONS of people has to be stored. This exceeds current capacities of available computers.
2. Response time will SUCK worse than current x-ray machine lines.
3. The false positive and false negative rates will soar with the addition of the other data to compare to.
Lets face it, this test failed MISERABLY. It is time to take our tax dollars elsewhere.
The number of shaky premises, false inferences, and unjustified conclusions in your post, is right up there with mainstream journalism.
Rather than show all these problems in formal logic, let me just point out one problem:
A humans ability to do facial recognition has nothing to do with a computer's inability to do it.
Also let me throw in the technical point that any parallel solution to the problem, can be emulated serially; it just will take more time and probably more hardware to do it.
1) They aren't any more effective than human personal, and unless they have an unrealistically low false-postive rate, they actually generate more work than they're worth
2) As anyone who's been to (or worked in) a DMV knows, there's basically 2 human reponses to computerized systems like this: a) you ignore the computer whenever it disagrees with you or b) you always obey the computer, no matter what. Unless you've got fantastically high sucess rates, b is exactly what the tinfoil croud is worried about (because this is even more important than getting the right address on your drivers license). A means that the computer system is a waste of time and money, which is the most common result if the system has low accuracy.
I rather suspect that the false positive rate would make this system impractical. If there were only one false identification per ten thousand people, then the false positives would significantly outnumber the true identifications. This has been pointed out to be the real weakness in biometric identification systems.
Of course, the 61% recognized were all Kennedys.
Taken from the ACLU web site:
According to the Logan report, which was written by an independent security contractor, "the number of system-generated false positives was excessive, and as a result, the operator's workload is taxing and strenuous, requiring constant undivided attention and periodic relief, which amounts to a staffing minimum of two persons for one workstation."
With quantum computer problems like this will be a thing of the past. We will just say "locate all criminals" and then the quantum computer will do it because they know things without actually knowing them. We will tell them to find the crimincals but they will have already found them, before the criminals even knew they were coming. Quantum computers will also be good for telling us what TV to watch and then rather than just watching the TV it will tell us whether the show we were going to watch was any good. Saves time that way. And then they never really had to make the show because the quantum computer already knew if we were going to like it so there is no real point in actually spending money to make the show. At least, this is my understanding of how quantum computers work. I think you can also do floating point math up to 10 digits too, but I wouldn't hold my breath.
Yeah, it was a bad idea to begin with, but let's give Boston and Tampa (Bay?) some credit.
The government saw once of its "law enforcement" / "war on terrorism" programs was ineffective and (gasp!) dropped it.
Isn't this exactly what we're *not* supposed to see from this bloated, non-responsive, heavy-handed bureaucracy/police-state that the libertarians/progressives bash all the time?
They had a trial run of a new, controversial idea and it didn't work. Isn't that exactly the sort of innovation and creativity people claim "big government" sorely lacks?
Joe
http://www.joegrossberg.com
I've taken the libery of fixing it. Now we can finally replace those hard-working security screeners:No thanks needed; I'm just glad I could help my country.
Recently there was a story in the press about a new rule for Canadian passorts: no smiles. Now we know why...to normalize the faces for facial recognition.
My dog does pretty well.
I've found that my posts don't format quite right w/o a sig.