Slashdot Mirror
Thousands in London Face Incorrect Benefit Cuts From Automated Fraud Detector (sky.com)
Thousands of people could soon be receiving letters threatening to cut off vital housing benefits as they face being incorrectly targeted by a new automated fraud detector. From a report: The government-backed London Counter Fraud Hub, developed by BAE, has been hailed a success after being trialled in four boroughs - Camden, Ealing, Croydon and Islington. Using vast quantities of data from millions of households, it is designed to target potential fraud cases involving the single person council tax discount, subletting in local authority housing and business rate relief and rating.
Ealing, the lead council for the project, found the automated elements of the system targeting single person discount fraud was 80% effective -- which is seen as an acceptable benchmark. With just over one million claimants of council tax single person discount in London, the London Counter Fraud Hub estimates it will detect around 40,000 fraudulent cases in the first year. Critics say the 20% error rate is unacceptable as around 8,000 people will receive letters wrongly accusing them of fraud.
Ealing, the lead council for the project, found the automated elements of the system targeting single person discount fraud was 80% effective -- which is seen as an acceptable benchmark. With just over one million claimants of council tax single person discount in London, the London Counter Fraud Hub estimates it will detect around 40,000 fraudulent cases in the first year. Critics say the 20% error rate is unacceptable as around 8,000 people will receive letters wrongly accusing them of fraud.
Nothing wrong with using an automated system that assumes conclusions. As long as you have
1) A viable system in place for remedying the failures. If you're unsure about the soft word in there, peg it at "better than youtube's"
2) The claimed conclusions are asserted tentatively. (The data in our system suggests) YOU LIED ABOUT X AND Z WILL HAPPEN (if not addressed).
Unfortunately we fuck up both. Privatize the savings, socialize the fuck up, charge forward trusting the data without hesitation, why stop at flags and alerts when you can have it execute outright.
This is just a high-level screening, false positives are not a big deal.
FTFA:
"We will not cancel anyone's council tax discount without giving them a fair say, which is why we are writing to them first. The council has experience of dealing with vulnerable people and others who need help communicating and will ensure we use the best channels of communication and provide support where necessary, as we do already," the spokesperson said.
"To err is human; to really foul things up you need a computer."
Table-ized A.I.
Automated fraud detection is fine.
But acting on a number which is basically a statistical guess, without some follow-up manual review is plain irresponsible.
The credit industry does this and regulation differs by country. For public services, the justice standards should be different.
I don't think we are hearing the whole story though. It is hard to believe that a 20% error rate is accepted by European standards of fairness.
I've always been bothered that all police in the US carry guns - where in general the police in the UK don't, at lest in the England of my youth. But make no mistake, there is a pernicious truth: despite unarmed Bobbies, England is a police state. Just look at the speed control on the roads. Variable Speed limits with long distance tracking, not a person in sight. I know there is a safety argument, but it is absolute control over people's movement. Literally. But that's not my point.
The real story is how a 20% Error Rate is even close to acceptable in any automated system, especially in an environment where addressing issues through bureaucracy is practically impossible?!?! On top of that every run will yield a new chance to be falsely accused so the threat always looms. Seeing this in my own USA is a much deeper fear for me than any terrorist activity, and 40% of the public will welcome it, including people who will be ruined by it. It's insane.
This is part of the thick edge of the wedge. Beware all.
Translation: There will be a premium rate phone number that goes to India. It will never be answered, not that it would be any use if it was.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."