UK Police Say 92 Percent False Positive Facial Recognition Is No Big Deal (arstechnica.com)

An anonymous reader quotes a report from Ars Technica: A British police agency is defending its use of facial recognition technology at the June 2017 Champions League soccer final in Cardiff, Wales -- among several other instances -- saying that despite the system having a 92-percent false positive rate, "no one" has ever been arrested due to such an error. New data about the South Wales Police's use of the technology obtained by Wired UK and The Guardian through a public records request shows that of the 2,470 alerts from the facial recognition system, 2,297 were false positives. In other words, nine out of 10 times, the system erroneously flagged someone as being suspicious or worthy of arrest.

In a public statement, the SWP said that it has arrested "over 450" people as a result of its facial recognition efforts over the last nine months. "Of course, no facial recognition system is 100 percent accurate under all conditions. Technical issues are normal to all face recognition systems, which means false positives will continue to be a common problem for the foreseeable future," the police wrote. "However, since we introduced the facial recognition technology, no individual has been arrested where a false positive alert has led to an intervention and no members of the public have complained." The agency added that it is "very cognizant of concerns about privacy, and we have built in checks and balances into our methodology to make sure our approach is justified and balanced."

Comments

Let's be positive

[hcs_$reboot]

Rate of 8% successful, meaning almost 1 in 10 people are correctly identified. Not that bad.

Re:Let's be positive

[ShanghaiBill]

Rate of 8% successful, meaning almost 1 in 10 people are correctly identified. Not that bad.

Indeed. If you are looking for a suspect in a city of a million people, and this system flags 10 people, and upon double checking you find that one of the ten is the suspect, then that is pretty darn good.

The false positive rate, by itself, tells you nothing about the usefulness of a test.

Re: Let's be positive

[c6gunner]

The problem is he's conflating the two facts of who commits more crime and why.

He didn't even talk about why, so how can he be conflating it with anything?

Black people commit more crimes in western nations because of a variety of reasons, the primary being socioeconomic status. Control for that factor across black populations and all of a sudden blackness isn't the thing to look for when predicting criminal behavior.

That's wonderful; even if you had incontrovertible evidence for this and could convince every single person on the earth that "socioeconomic status" was the one and only reason for the difference it would still have absolutely nothing to do with what he was talking about.

However, if you really want to go off topic, consider the fact that your explanation says nothing about why there's such a big difference in the "economic status". You haven't actually explained anything, you've just pointed out a correlation without demonstrating a root cause. It's like saying "the reason country X has more car accidents is because people drive worse". Well thank you Dr. Science.

Re: Let's be positive

[Anonymous Coward]

The problem is he's conflating the two facts of who commits more crime and why.

Black people commit more crimes in western nations because of a variety of reasons, the primary being socioeconomic status. Control for that factor across black populations and all of a sudden blackness isn't the thing to look for when predicting criminal behavior.

The problem with your "paint" theory or "blank slate" theory is that there are no successful prosperous black-run areas that are safe pleasant places to live. Including when blacks control all of the political and economic activity. Haiti is a good case in point. It was a productive agrarian economy with things like public sanitation and law enforcement while the French ran it. When the blacks intercepted a shipment of muskets, ammo and powder they overthrew the French and had their own version of the American Revolution (except they slaughtered every last French man, woman and child indiscriminately, not confining themselves to military targets like the Founding Fathers did against the Redcoats). Not long after it became a wasteland where people literally shit in the streets and contaminate their own drinking water.

If you want to understand why racial profiling happens then have a look at the facts for yourself. This includes nations that never oppressed black people.

What you call "racism", I call pattern recognition. Isn't it interesting that the black violent crime rate has gone UP ever since the Civil Rights Movement, affirmative action, and equal opportunity? One would think that if white oppression was the problem, reducing that would have a positive impact. It's as though Jim Crow served to protect whites from an inherent trait that our ancestors recognized. Tell me, did you really believe that white flight is caused by an aversion to melanin? No. It's caused by black violence. Black males are about 6-7% of the US population yet account for 51% of all solved murders. They are 12 times more likely to murder a white person than vice-versa. Still, mostly they murder other black males, but tell me, would you like to live in such a place? See the FBI crime stats for yourself.

Isn't it odd how South Africa has seen a serious economic decline and a simultaneous increase in crime ever since apartheid ended? Left to their own devices, this is what blacks create. The examples are numerous. These are facts. I actually would prefer it not be true, that it's just skin color, but it isn't and I refuse to lie to myself when it comes to choosing a safe prosperous place for my family. The person who does otherwise is either stupid or insane.

Re: Let's be positive

[Cederic]

You do realise that 'black' is merely a label for a range of skin colours, races and (at a global level) cultures?

Me, I'd happily live in Morocco. It's fucking awesome.

It's as though Jim Crow served to protect whites from an inherent trait that our ancestors recognized.

Ah, sorry. My bad, I didn't realise I was replying to an idiotic fuckwit.

Re: Let's be positive

[Cederic]

The UK police have been using a system of racial profiling that says "if you are black, you are probably in a gang" - however, investigation reveals that the system is mostly wrong.

It is wrong, and 'stop and search' does seem to be disproportionately applied to certain skin colours (although I've been stopped and searched - twice in five minutes). Similarly the knife crime in London is heavily skewed to certain demographics - but as https://www.theguardian.com/co... suggests, the primary driver is not race.

Me, I'm distressed that these kids don't feel they have better options. That's a gender issue, not a race one.

Re: Let's be positive

[Anonymous Coward]

Hmm, but again, I don't think it has to do with skin colour but with ethnic upbringing, the set of core values, the level of pride and its worth against the worth of other humans life. E.g. Afghanistan is a country of virtually all-white people. Yet, of all the lethal violent crimes among recent migrants in Europe, most of them is done by Afghan young men. Or people from Balkans, or people from most parts of former Soviet Union, so often actors in crimes, even organized crime, also all white.

Black people either come from African tribal cultures where inter-tribal tensions and violence is a norm, or are descendent of slaves who were systematically humiliated and now they have this urge to overcompensate by exerting their self-esteem and dangerousness.

Haiti had its revolution, but it din't have its merchant and intellectual elite to form the leadership and successful state organization. Revolutions are executed by revolting masses, but successful ones are usually prepared, supported, and finally, exploited by social stratum just beneath the previous (and subsequently beheaded) top. When all the leaders a country has at its disposal are warrior higher ranks, it will get ... wrong order of priorities, so to speak, as well as very literal, as in "bloody", power struggle. But then again, in history of many similar nations there has been a way out, provided one strong (tyrant) leader emerges at the top and makes proper moves to create new elite out of selected individuals in its young generation. However, unlike some other (white) countries, Haiti neither had no access to universities with long tradition, nor skilled ship-builders and sailors to arrange a merchant fleet for export of its produce. Finally, after freedom comes, the worst of jobs, jobs only slaves would do, will be left undone. Ergo, your example is not representative.

Re: Let's be positive

[Anonymous Coward]

When I hear such claims I like to use an example that perhaps highlights the problems itself better:
Take a baby and isolate it away, perhaps in a dungeon (or elsewhere horrible) or in the wild with wolves, and wait until it has fully matured.
Now take that person out of that isolation, give them 1 million dollars and toss them out into the real world.
What do you expect will happen to that person?

Yet this is EXACTLY what we did with people and countries. Threw them out, gave them money (if at all) to dampen our shame and left them to fend for themselves.
'Pfft you're free now, time to start acting correctly...'
We were not the best examples were we? Think about how the were treated... that is all they knew.

You are free, but we do not want you around us. We will not include you into our culture nor will we give you the same chances. We are born on the higher steps and act like we had to work to reach that height, while at the same time laughing at others that have to fight hard for every little step. And every step they do climb we not only complain why they only got one step, and then beat them back down.

Fact is, we all did sh1t to live in the wealth we have. We should be ashamed of ourselves for b1tching about others.

This is not only about people of color or other countries. This happens in OUR countries as well. Kids born into poverty, regardless of heritage or skin color, will face similar hardships.

This world is not perfect and not all are examples. Rich or poor have many bad examples. But those born with a golden spoon up their rears, should shut up about how hard life is.

Re: Let's be positive

[drinkypoo]

Including when blacks control all of the political and economic activity.

Where is that?

Haiti is a good case in point.

Haiti has been subject to American influence. America controls much of the economic activity. Try again!

Re:Let's be positive

[eth1]

Rate of 8% successful, meaning almost 1 in 10 people are correctly identified. Not that bad.

The thing is, if the false positive rate is that bad, I would argue that a "match" isn't good enough to constitute reasonable suspicion/probable cause for an arrest. If they stopped me for an "intervention", based on a system that bad, I'm either free to go, or under arrest. If they won't let me leave, I'm under arrest, and would consider filing a suit over it (mainly to make horribly inaccurate systems like this less attractive to LE).

Intermediate false positive rate

[larryjoe]

despite the system having a 92-percent false positive rate, "no one" has ever been arrested due to such an error

I may have concerns about the civil liberty impact of broad-net surveillance systems in general, but the algorithmic deficiencies of this particular system are portrayed incorrectly in this article. I.e., the front-end of the system (the facial recognition system) has a 92% false positive rate, but together with the post-processing in the back-end, the total system has a false-positive rate of 0%. This is similar to saying that the object detection failure probabilities for a ADAS system need to be viewed in the context of the entire system, and it's the performance of the total system that is significant.

Re:Intermediate false positive rate

[Zocalo]

Agreed; the part of the summary where it says "no individual has been arrested where a false positive alert has led to an intervention" implies that some of these false positives are resolved by an officer doing further checks, which might just be comparing the CCTV image with a mugshot and deciding it's not a match. Privacy issues of the surveillance state aside, as far as the member of the public that was incorrectly flagged is concerned, I guess that's no harm, no foul because they are none the wiser.

However, it then goes on to claim that "and no members of the public have complained", so presumably at least some of those manual checks also including getting an officer to stop some of the targets and verify their ID, otherwise the possibility of complaints wouldn't even be an issue. Without data on that split it's kind of hard to gauge the effectiveness of the system as a whole, and it would also be useful to have the other side of the coin; how many of the 173 people that were arrested as a result of the system does the South Wales Police dept. think might have otherwise been overlooked in the crowds? If that's a significant fraction of those 173 arrests, then that seems like it's a pretty good tool to help maintain law an order in a public space to me, at least until the criminals realise they need to avoid such places and/or provide a clear image to the cameras.

Re:When it comes to criminals and esp terrorists

[Anonymous Coward]

I'd rather err on the side of false positives than false negatives (which let them slip away). A minor inconvenience is worth the extra security by far.

Exactly!

A few innocent lives may be lost, but that's a small price to pay for my peace of mind.

Bad maths or fishing

[Anonymous Coward]

2,470 alerts - 2,297 false positives = 173 true positives.
>450 people arrested from "facial recognition efforts".

Either that means there were >277 false arrests due to facial recognition, or they are counting arrests due to "facial recognition efforts" as also including the results of things they found when the searched people based on those false positives.

Since they claim "no one has ever been arrested due to such an error", so this means that both that the number of successful arrests has been inflated to make the system look more useful, and that the system's primary function is to justify illegal searches.

Re:Bad maths or fishing

[thegarbz]

Or (and I'm going out on a limb here) you're getting data from two different sources which haven't been normalised for time, duration, or public relations content.

Re:When it comes to criminals and esp terrorists

[110010001000]

Sure. But I am talking Benjaman Franklin, not Benjamin Franklin.

Working as intended

[GrumpySteen]

Catching criminals is a side effect. The main purpose is to create justification to investigate anyone they want.

Screening

[jma05]

A percentage, without the context of use, is meaningless.

They might be using them for screening, to focus human evaluation. If so, that means that it is ultimately the cop that makes the decision, not the system. This is how today's AI is meant to be used - as a cognitive aid.

It is fairly common for screening tests in medicine to have high false positive rates. That is OK. They are just meant to narrow down the search space for more expensive/invasive confirmatory tests. Given that the incidence of criminal targets will always be a tiny percent of the corpus, it is very difficult to have tests with high true positive rate. That is quite normal for general tests, in general.

The questions that are relevant are:

1. Are the police able to better solve crime with the aids?
2. Is the test too expensive for the said improvement?
3. What are the rates of negative outcomes (like a wrongful arrest) and..
4. What do we, as a society, consider to be acceptable thresholds?

Re: Using Technology for the Wrong Purpose

[fafalone]

Well I for one don't think mass facial recognition should be used at all, because anyone who thinks it won't be abused just hasn't been paying attention.