Amazon's Facial Recognition Wrongly Identifies 28 Lawmakers, ACLU Says (nytimes.com)

Representative John Lewis of Georgia and Representative Bobby L. Rush of Illinois are both Democrats, members of the Congressional Black Caucus and civil rights leaders. But facial recognition technology made by Amazon, which is being used by some police departments and other organizations, incorrectly matched the lawmakers with people who had been arrested for a crime, the American Civil Liberties Union reported on Thursday morning. From a report: The errors emerged as part of a larger test in which the civil liberties group used Amazon's facial software to compare the photos of all federal lawmakers against a database of 25,000 publicly available mug shots. In the test, the Amazon technology incorrectly matched 28 members of Congress with people who had been arrested, amounting to a 5 percent error rate among legislators. The test disproportionally misidentified African-American and Latino members of Congress as the people in mug shots.

"This test confirms that facial recognition is flawed, biased and dangerous," said Jacob Snow, a technology and civil liberties lawyer with the A.C.L.U. of Northern California. Nina Lindsey, an Amazon Web Services spokeswoman, said in a statement that the company's customers had used its facial recognition technology for various beneficial purposes, including preventing human trafficking and reuniting missing children with their families. She added that the A.C.L.U. had used the company's face-matching technology, called Amazon Rekognition, differently during its test than the company recommended for law enforcement customers.

Comments

Well, it was half right...

[magusxxx]

...they just haven't been arrested yet. :D

AI sometimes isn't perfect either

[foxalopex]

I think the whole idea of using face recognition is to cut the amount of work required by a detective to search through thousands of pictures. I'm sure the final step would be for a real person to verify the matches to see if there's false positives. The AI in this case would likely be setup to tend to produce false positives rather than outright missing matches because not being able to find anything is worrysome compared to finding a few false positives. You would hope the cops arn't crazy enough to start arresting people based entirely on the matching system and at least look at the profiles to confirm.

Re:AI sometimes isn't perfect either

[ranton]

I'm sure the final step would be for a real person to verify the matches to see if there's false positives. The AI in this case would likely be setup to tend to produce false positives rather than outright missing matches because not being able to find anything is worrysome compared to finding a few false positives. You would hope the cops arn't crazy enough to start arresting people based entirely on the matching system and at least look at the profiles to confirm.

This is exactly correct, and why these statements from the ACLU are ridiculous. Would they rather the police just be looking for any tall black guy with a sweatshirt in the area? This type of technology simply provides more information to the police, but it still takes actual policemen and prosecutors to decide who is a real suspect and who should be charged.

Re:AI sometimes isn't perfect either

[jeff4747]

You would hope the cops arn't crazy enough to start arresting people based entirely on the matching system and at least look at the profiles to confirm.

What about recent law enforcement activities have given you any reason for this hope?

You "match", you get arrested. And held until you can pay the bail for your "crime", or decide to plead to a lesser charge for time served.

Meanwhile, your life is completely destroyed while you're in jail, because you can't work, can't pay your bills, lose your house because you can't pay the mortgage/rent, can't care for your kids, and so on. So there's a ton of pressure to plead to something just to stop the destruction. And when you do, the cops have "solved" the crime and look good.

Re:AI sometimes isn't perfect either

[Wycliffe]

The mere fact that innocent citizens show up on the radar at ALL for police trying to solve a crime is very troublesome.

You want to absolutely minimize false positives.

I disagree. I think you should set up the AI to always produce false positives and probably hide the percentage of the match as well. Just like a lineup it should always return the top 10 results sorted randomly regardless of how closely they match. That way the cops don't start relying on it as something that it isn't.

Re:AI sometimes isn't perfect either

[bigpat]

I'm sure the final step would be for a real person to verify the matches to see if there's false positives. The AI in this case would likely be setup to tend to produce false positives rather than outright missing matches because not being able to find anything is worrysome compared to finding a few false positives. You would hope the cops arn't crazy enough to start arresting people based entirely on the matching system and at least look at the profiles to confirm.

This is exactly correct, and why these statements from the ACLU are ridiculous. Would they rather the police just be looking for any tall black guy with a sweatshirt in the area? This type of technology simply provides more information to the police, but it still takes actual policemen and prosecutors to decide who is a real suspect and who should be charged.

Yes and no. So when I have run image recognition through a neural net I get a percentage match... so it depends what the threshold for a match is set at. Is 65% considered a match or 95% or 99.9%? The devils in the details and I could see the percentage being obscured from the end user to the point of police and the courts treating it as a binary value rather than with any relative degree of certainty because the police and the courts want to be right and are under time constraints to be right and move on.

So depending on the percentage match I could see people in the same racial group being matched... but would a court issue a warrant based on someone saying they are in the same racial group... because I could see the police saying that "they were a match using facial recognition" and the court just rubber stamping that because it obscures the real underlying lack of certainty. There is a real danger of abuse depending on how facial recognition is used (like any tool), but neural net algorithms are especially prone to obfuscation.

On the other side, people are often terrible witnesses and have their own underlying lack of certainty that can be obscured without the reproducible and adjustable nature of image recognition. People are often wrong in their recollection and many people have gone to jail because of wrong identification by witnesses, sometimes even multiple witnesses.

In other words their is uncertainty no matter what... the good and the bad news with AI is that you can begin to quantify that uncertainty. So image recognition is good news for improving accuracy over human perception, but bad news if it is either misunderstood or willfully abused to create the misconception of 100% accuracy.

Re:AI sometimes isn't perfect either

[Wycliffe]

When you consider the large crowds in the public spaces where this system is likely to be deployed, a 5% false positive rate would result in unmanageable numbers to verify. -E.g. Times Square sees 300,000 people a day movement, resulting in 15,000 false positives a day. Even a 1% false positive rate would be too high, especially considering the cost in civil liberties involved to those falsely flagged.

They aren't going to arrest 15000 people a day so there is no "cost in civil liberties involved to those falsely flagged" nor are they going to arrest 1000 people but it could help them quickly look at those 1000 people from a distance versus having to do the impossible job of trying to look at all 300k people. A large false positive is actually probably a good thing. If the false positive is too small like say only 0.01% then the cops might be tempted to arrest all 30 of those people without doing due diligence.

Garbage In, Gospel Out!

[Thud457]

I find it hard to believe that only 28 members of Congress are criminals. This AI needs to go back for reeducation.