UK Police Say 92 Percent False Positive Facial Recognition Is No Big Deal (arstechnica.com)
An anonymous reader quotes a report from Ars Technica: A British police agency is defending its use of facial recognition technology at the June 2017 Champions League soccer final in Cardiff, Wales -- among several other instances -- saying that despite the system having a 92-percent false positive rate, "no one" has ever been arrested due to such an error. New data about the South Wales Police's use of the technology obtained by Wired UK and The Guardian through a public records request shows that of the 2,470 alerts from the facial recognition system, 2,297 were false positives. In other words, nine out of 10 times, the system erroneously flagged someone as being suspicious or worthy of arrest.
In a public statement, the SWP said that it has arrested "over 450" people as a result of its facial recognition efforts over the last nine months. "Of course, no facial recognition system is 100 percent accurate under all conditions. Technical issues are normal to all face recognition systems, which means false positives will continue to be a common problem for the foreseeable future," the police wrote. "However, since we introduced the facial recognition technology, no individual has been arrested where a false positive alert has led to an intervention and no members of the public have complained." The agency added that it is "very cognizant of concerns about privacy, and we have built in checks and balances into our methodology to make sure our approach is justified and balanced."
In a public statement, the SWP said that it has arrested "over 450" people as a result of its facial recognition efforts over the last nine months. "Of course, no facial recognition system is 100 percent accurate under all conditions. Technical issues are normal to all face recognition systems, which means false positives will continue to be a common problem for the foreseeable future," the police wrote. "However, since we introduced the facial recognition technology, no individual has been arrested where a false positive alert has led to an intervention and no members of the public have complained." The agency added that it is "very cognizant of concerns about privacy, and we have built in checks and balances into our methodology to make sure our approach is justified and balanced."
Rate of 8% successful, meaning almost 1 in 10 people are correctly identified. Not that bad.
Slashdot, fix the reply notifications... You won't get away with it...
It sounds like the UK police hired needful doers again and as usual need to save face more than they need to protect English citizens.
...all those damn muslims.
Ask questions later.
If you understand that your facial recognition has a 92% false positive rate, it's fine, as long as you use it that way.
Look at what the numbers actually mean. The system tagged 2,470 possibles and 2,297 were false positives (so 173 hits). There's about a million people in South Wales, so probably 50,000-100,000 people with the same sex, build, and broad age range as the wanted suspect. You're looking for 173 people in a field of 100,000 and your automated system reduced the size of the field to 2,470. That's not a bad start.
It is really hard to figure out a person by description, which is what police do under normal circumstances.
At least this narrows it down a bunch.
Not sure that big brother is such a great thing, but the false positive rate doesn't seem to be a real concern to me.
despite the system having a 92-percent false positive rate, "no one" has ever been arrested due to such an error
I may have concerns about the civil liberty impact of broad-net surveillance systems in general, but the algorithmic deficiencies of this particular system are portrayed incorrectly in this article. I.e., the front-end of the system (the facial recognition system) has a 92% false positive rate, but together with the post-processing in the back-end, the total system has a false-positive rate of 0%. This is similar to saying that the object detection failure probabilities for a ADAS system need to be viewed in the context of the entire system, and it's the performance of the total system that is significant.
For police work, identifying suspects, false positives only affect the overhead portion - rejecting someone identified. If however it had a false negative, then it would be an issue as it would let people who should be suspects go away free. For the moment, as long as they aren't looking for too many people , false positives just allow them to remind the LEO fearing folk that there is law and order in the land.
What is dangerous is that if the rate does not improve, and you have 10/% of their population doing crimes, then they would have to constantly investigate and examine 100% of their population, which they won't be able to have the mass of staff to do.
I'd rather err on the side of false positives than false negatives (which let them slip away). A minor inconvenience is worth the extra security by far.
Exactly!
A few innocent lives may be lost, but that's a small price to pay for my peace of mind.
but still being used for the same purpose: to justify an illegal fishing expedition.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
Overall, I'll just for a second pretend that any of this is what anyone wants (which it is not). They at least only require a software upgrade to improve facial recognition as all the hardware is currently present. Were they to acquire an expert in neural networks and image recognition they have every possibility of bringing this number up to much higher accuracy levels over time. Overall, a bad system, but only a bad system at this moment in time. This does not mean that as the years roll on and software improvements are made that the system will not get better and better.
Terrifying really when you think of this system running at full possibility a few years down the road.
"It is better that ten guilty persons escape than that one innocent suffer."
- Sir William Blackstone.
"I'm more concerned with bad guys who got out and released than I am with a few that in fact were innocent."
- Dick Cheney.
we have built in checks and balances into our methodology to make sure our approach is justified and balanced
That's some sickening shit! How do we combat this when nobody resists? On the contrary, it wins more votes!
"We"? Don't checks and balances typically require outside stakeholders to be directly involved?
Table-ized A.I.
2,470 alerts - 2,297 false positives = 173 true positives.
>450 people arrested from "facial recognition efforts".
Either that means there were >277 false arrests due to facial recognition, or they are counting arrests due to "facial recognition efforts" as also including the results of things they found when the searched people based on those false positives.
Since they claim "no one has ever been arrested due to such an error", so this means that both that the number of successful arrests has been inflated to make the system look more useful, and that the system's primary function is to justify illegal searches.
"Those who would give up essential Liberty, to purchase a little temporary Safety, deserve Safety." - Benjaman Franklin
are they being buried in data and information? And by the time they sort through it all the intelligence maybe meaningless. Having to much chaff mixed in with the grain your looking for can be a bigger problem overall.
;)
Just my 2 cents
Benjamin Franklin was a traitor to the crown. His words carry no weight in the United Kingdom, chap.
Sure. But I am talking Benjaman Franklin, not Benjamin Franklin.
Here is a proposal for a simplified facial recognition algorithm. It features 0% false negative, at the price of an acceptable false positive rate
bool is_suspect(char *picture, size_t picture_len) { return true; }
"That's much better than officers!"
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
Catching criminals is a side effect. The main purpose is to create justification to investigate anyone they want.
see how that goes next time you get pulled over.
He MUST have violated a law at some point! If not, we neew more laws, because the computarrrr cant be wrong!
how do those boots taste?
Another system in current use for doing similar police work is to make public calls for information that might be helpful to a case or broadcast sketches or grainy videos of suspects and ask for the public to call in. What percentage of those calls are false positives? My bet is it is vastly higher than 92%.
I'm more interested in their claim that no one has been arrested due to a false positive. That's nearly impossible to completely avoid without the use of a facial recognition system. Has the UK found a new system that allows them to only arrest guilty people without the need for a costly legal system?
You just earned a fix-it ticket for your defective speedometer. Check by the police garage next Wednesday at 2:30pm to prove you've gotten your speedometer repaired, or pay a stiff fine.
You just earned a fix-it ticket for your defective speedometer. Check by the police garage next Wednesday at 2:30pm to prove you've gotten your speedometer repaired, or pay a stiff fine.
Nice. In my jurisdiction (US), speeding is a strict liability offense.
Doesn't matter if your speedometer is broken. If your car was traveling faster than the limit, you'll pay the speeding ticket; it does not matter why.
I was in court to challenge a vehicle registration ticket, and I was waiting my turn.
The judge was hearing another case about a speeding ticket, and the young woman had proof in hand from her auto-mechanic that the speedometer was faulty and repaired.
The judge told her that it didn't matter, she was still to pay the ticket.
> how many of the 173 people that were arrested as a result of the system does the South Wales Police dept. think might have otherwise been overlooked in the crowds? If that's a significant fraction of those 173 arrests
Based on my experience, I'd estimate that approximately zero would have just been noticed by a cop saying "hey that guy looks like someone who has a warrant". There are a LOT of wanted criminals, far too many to memorize each face. Very few fugitives are caught that way.
They do tend to get caught eventually when they screw up while committing further crimes. They get busted for one offense, don't show up to court, and go back to selling drugs or breaking into cars or whatever their thing is. Eventually they screw up, get unlucky, and get caught. That's when the warrant matters - after they get caught again.
May. 4, Juche 107 (2018) Friday
Unity Based on Independence Guarantees Victory of Cause of Independence
The unity of the independent forces against the imperialists provides a sure guarantee for accomplishing the human cause of independence.
The popular masses' struggle for realizing the desire for independence is an arduous struggle against the imperialists and all sorts of reactionaries putting a brake on it.
The confrontation between the independent forces and the hegemonic forces inevitably accompanies the confrontation of strength.
The tremendous might pushing forward the struggle for independence against the imperialists lies in the unity of the independent forces.
Unity provides a fundamental way of strengthening the driving force for accomplishing the cause of global independence.
The respect for sovereignty is a fundamental principle to be maintained by the progressive people of all countries in their struggle against the imperialists.
Independence is the lifeline of a country and nation and also the lifeline of humankind.
The imperialists are dregs of history who have outlived their days but they are reluctant to come down from the stage of history of their own accord.
Today the imperialist reactionaries leave no stone unturned to stamp out the forces for independence against imperialism in collusion.
They are trying desperately to divide and disintegrate the forces for independence against imperialism including the non-aligned movement and establish their order of domination.
This has seriously violated and encroached upon the sovereignty of countries and nations and harassed the world peace and security.
The situation urgently demands that all the countries advocating independence strengthen unity and solidarity and strenuously wage the struggle against the imperialists by their concerted efforts.
Ri Hyo Jin
Compared with the total attendance of 65,842 (wikipedia), looking through 2,470 to double check them isn't that bad.
The way investigative policing works is you have numerous leads, and you follow up on them, and most end up asv dead ends, but hopefully some bear fruit.
A false positive is a lead that didn't work out.
First of all. I know someone who files complaints regularly and has good reasons to do so because his job puts him in contact with police and they routinely abuse him and others. Do you know where these complaints usually go? In the garbage. Humorously one cop, his boss, and prospector conspired to take retaliatory action against him for *FILING* a complaint. This was on the argument that the cop contested the account of what happened and basically presumed cops are always truthful (despite that it was all filmed) they put out a warrant for his arrest (this is after he went into the station and I filmed the whole complaint taking process). I was there when another set of police drew arms on him and made the arrest too. If it wasn't for the outrageousness of the case and being a high profile individual no action would have been taken against either the abusive cop that he was filing a complain on nor the conspiracy that occurred afterward. One of the best civil rights lawyers in New England took up the case and after that it resulted in an investigation into multiple officers who were the persons that the original complaint was filed against. The case was dropped only because the officer who was complained against lost his job because an investigation got started for other corruption that the officer and his associates were doing. If it had not been for the retaliatory arrest the lawyer would not have been involved and the complaint would have been ignored ( it was the lawyer really that caused closer inspection of the officer and his associates over other unrelated matters that led to the officers firings). But the point of the story here is the police are just another gang, corrupt, and liars.
I have lots of stories about police being low lifes and nothing but a bunch of lairs. This reminds of how the police here hand out questionnaires at unconscionable suspiciousness checkpoints on the guise of stopping drunk drivers (but when in fact patrolling is MORE effective according the police's own statistics and has admitted such) that explicitly lead to 90% of people are happy that we setup suspiciousness drunk driving checkpoints completely ignoring that there are a sizable number of continuing protests at every checkpoint in that area, the real reason they like doing them is because of the overtime pay they get from doing so, they can largely get away with ignoring the law and rules that protect us from abusive policing practices, etc. And the reason for the survey was the police wanted to be able to go into the state house to kill any bill proposing to undo the practice which only became legal because of traitors in the supreme court who sold out our freedom on bogus arguments of "safety". In this state we merely wanted to re-affirm what the constitution has always been interpreted to mean (ie suspiciousness checkpoints are illegal) and in fact still are even under the courts ruling with the exception of drunk driving and there not suppose to look for other shit- but do- anyway. Give the police an inch and they'll take your life. Literally.
For instance under any other circumstance the police need a retain to temporarily detain you. They must have articulable suspicion of a crime. People confuse this with reasonable suspicion but it is articulable suspicion. In other words walking home late at night a cop might think it is reasonable for them to stop you. Maybe it is. It's not a crime though so there is no articulable crime to point to that he can say "I think he did X". If on the other hand of the person had a paint can he was holding and another officer on the radio had previous said "look out for suspect with a paint can coming from direction X as someone just gratified so and so's house" then there would be reason to believe that a particular crime had been committed and reason to detain that person exists. With a suspiciousness detainment cops just ask questions and look for something upon which to arrest people at random. They mostly catch people for petty things. Like seat belts, license plate frame
A percentage, without the context of use, is meaningless.
They might be using them for screening, to focus human evaluation. If so, that means that it is ultimately the cop that makes the decision, not the system. This is how today's AI is meant to be used - as a cognitive aid.
It is fairly common for screening tests in medicine to have high false positive rates. That is OK. They are just meant to narrow down the search space for more expensive/invasive confirmatory tests. Given that the incidence of criminal targets will always be a tiny percent of the corpus, it is very difficult to have tests with high true positive rate. That is quite normal for general tests, in general.
The questions that are relevant are:
1. Are the police able to better solve crime with the aids?
2. Is the test too expensive for the said improvement?
3. What are the rates of negative outcomes (like a wrongful arrest) and..
4. What do we, as a society, consider to be acceptable thresholds?
A few innocent lives may be lost, but that's a small price to pay for my peace of mind.
This is, quite literally, the rationale for every form of law (and law enforcement) which has ever existed or will ever exist. Even in the UK, where "police don't carry guns", a few innocent die and many innocents end up rotting in jail. We as a society accept that cost because we understand that no system is perfect but almost any system is better than none. We can try to reduce the number of innocents killed or otherwise injured in the pursuit of justice, but we will never get that number down to zero. So yes, a few innocent lives may be lost, but that's a small price to pay for our piece of mind.
92% false positives is not a problem IF they require all matches to be checked by a human being for taking any action against the person matched. And don't have the AI matches confirmed by somebody with face blindness (prosopagnosia) like me, either!
I've abandoned my search for truth; now I'm just looking for some useful delusions.
Yeah, right.
This is exactly using technology for something it is completely unsuited.
Facial recognition is useful as second or third-factor authentication of a small and clearly defined user base. Like checking the face of a person wanting to pass a security door whilst the same person is in possession of a RFID badge. Not only do you match against a smallish set of people who "shall pass", but against the very small set of people who may pass with that specific RFID badge, exactly one, that is. And in this case, security is immensely increased by facial recognition.
"The more prohibitions there are, The poorer the people will be" -- Lao Tse
The "crown" is a traitor to the people of the Republic of Britain. Toss the royals out on their ears.
Time for a revolution, chap.
HAL: Let me put it this way, Mr. Amer. The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error.
There is such a thing as police in the UK but not necessarily UK Police. If there was such a thing, they would be looking to protect all of the citizens of the UK rather than only the English.
I hate to be a pedant but the article focuses on an instance in Wales which is an entirely different country to England, albeit with the same legal system.
I'd rather err on the side of false positives than false negatives (which let them slip away). A minor inconvenience is worth the extra security by far.
Exactly!
A few innocent lives may be lost, but that's a small price to pay for my peace of mind.
Because no lives are lost if terrorists aren't identified and caught? I'll take the peace of mind and nobody gets blown up package thanks.
And hold you for questioning, they haven't arrested you. But you still have to be sent to the police station and held in a cell for a day. They still haven't arrested you, they only got you in to question. And if it was because they had no reason other than a system that produced 92% false matches, would YOU think that the detention (not arrest!) for a day or so would be fine and dandy????
Do you think they investigated the result? Or do you think they just detained (not arrested!) all of them for questioning and released them without charge (still not arrest!)? And for c6gunner, there is no difference and that is the problem. At least before you could pont to someone who could be punished for fucking it up, this way you have nobody to blame and nobody to correct to avoid it in the future. Malice shows that it can be fixed by removing the malice,but incompetence is immune to change by its nature.
The worst thing about technology is relying on it when it doesn't work well. Then defending it even when its obvious it doesn't work. Probably because you spent ridiculous amounts of resources and money on it.
just like a monitoring systems that beeps every few seconds and alerts on eveything which is in some cases important, it will be ignored by its users.
in that case, why keep it running, just turn it off.
On a long enough timeline, the survival rate for everyone drops to zero.
Benjaman Franklin... wasn't he one of Jamaica's founding fathers?
2297 / 2470 = 93%, not 92%
When 92% of the positive results are false, that is not a 92% false positive rate. Look it up.
The police website explains very well how 92% of initial positives being false goes together with 0% false identification leading to arrest:
The operator considers the initial alert and either disregards it (which happens on the majority of cases) or dispatches an intervention team as the operator feels that the match is correct. When the intervention team is dispatched this involves an officer having an interaction with the potentially matched individual. Officers can quickly establish if the person has been correctly or incorrectly matched by traditional policing methods i.e. normally a dialogue between the officer/s and the individual. If an incorrect match has been made officers will explain to the individual what has happened and invite them to see the equipment along with providing them with a Fair Processing Notice (which can be found on our SWP website).
So they have human face recognition, and they can ask people for ID.
Also, being in a police search does not mean that they identify you "as a potential criminal" as Wired would have it. Everybody is a potential criminal. Investigating people because they look like somebody with a search warrant is just normal police work, no matter if with a computer or not
Being wrong 9 times out of 10 means they're aware it's not safe to arrest someone just 'cos the computer says they look shifty, Consider the reverse: if its right 9 times out of 10, when the computer doesn't like their look they'll be in the clink before the ink's dried on the printout.
This way even the cops should know not to trust it.
The reason no-one has been arrested is because the Police use a mythical status of "detained" to make people think they're arrested without the police incurring the liability of wrongful arrest. If ever a policeman tells you you're "detained", try wandering off, they won't turn it into an arrest.
Used to be a 90 percent false positive rate. My guess is they don't understand what they're doing, but like arresting people for not being white.
-- Tigger warning: This post may contain tiggers! --