Slashdot Mirror
Software 'No More Accurate Than Untrained Humans' At Predicting Recidivism (theguardian.com)
An anonymous reader quotes a report from The Guardian: The credibility of a computer program used for bail and sentencing decisions has been called into question after it was found to be no more accurate at predicting the risk of reoffending than people with no criminal justice experience provided with only the defendant's age, sex and criminal history. The algorithm, called Compas (Correctional Offender Management Profiling for Alternative Sanctions), is used throughout the U.S. to weigh up whether defendants awaiting trial or sentencing are at too much risk of reoffending to be released on bail. Since being developed in 1998, the tool is reported to have been used to assess more than one million defendants. But a new paper has cast doubt on whether the software's predictions are sufficiently accurate to justify its use in potentially life-changing decisions.
The academics used a database of more than 7,000 pretrial defendants from Broward County, Florida, which included individual demographic information, age, sex, criminal history and arrest record in the two year period following the Compas scoring. The online workers were given short descriptions that included a defendant's sex, age, and previous criminal history and asked whether they thought they would reoffend. Using far less information than Compas (seven variables versus 137), when the results were pooled the humans were accurate in 67% of cases, compared to the 65% accuracy of Compas. In a second analysis, the paper found that Compas's accuracy at predicting recidivism could also be matched using a simple calculation involving only an offender's age and the number of prior convictions.
The academics used a database of more than 7,000 pretrial defendants from Broward County, Florida, which included individual demographic information, age, sex, criminal history and arrest record in the two year period following the Compas scoring. The online workers were given short descriptions that included a defendant's sex, age, and previous criminal history and asked whether they thought they would reoffend. Using far less information than Compas (seven variables versus 137), when the results were pooled the humans were accurate in 67% of cases, compared to the 65% accuracy of Compas. In a second analysis, the paper found that Compas's accuracy at predicting recidivism could also be matched using a simple calculation involving only an offender's age and the number of prior convictions.
Only bad programmers/designers.
Slashdot, fix the reply notifications... You won't get away with it...
It seems obvious that someone with more relapses in the past will also be more likely to do it again. However, I will assume that at that point, a judge wont allow for bail anyway so if this is about people with three or less offenses on their record, I'd imagine that ONLY going by the criminal history is going to be inaccurate no matter who or what is looking at it.
Isn't this more a case of bad data as opposed to bad programming? Because "no more accurate than an untrained person" implies pure chance.
Tl;Dr Single old program tested in situation vendor says is inaccurate use of software, software doesn't work well. Thus all programs will forever be terrible at this task and these computer guys should give up and do something useful. Like writing headlines for news sites!
Isn't this precisely what you would expect when the information gathered to make the decision isn't influential enough on the outcome. It says they have 137 variables, which were as useful as 2. It suggests that the additional variables are either unrelated to the outcome, or are strongly related to the 2 suggested such that either way they provide no additional accuracy.
They are trying to solve the wrong problem. Rather than trying to quantify people, the solution to people reoffending is to provide better support to everyone. Stop wasting money on software and start investing in programmes that help reform offenders.
Reform programmes are also a much better way to evaluate people, because their progress in the programme is much easier to measure and requires them to meet goals that change their behaviour and future life chances. That's why sensible systems hand out a sentence which can then be reduced through participation and good behaviour.
const int one = 65536; (Silvermoon, Texture.cs)
SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
No. The problem is that people have realized the software is racist. What happens is this:
Black citizens tend to get more minor criminal issues than white ones because of institutional racism. Then this software sees that a black man has two citations for, say crossing the street away from a crosswalk, while the white man does not. So it gives him a higher risk of recidivism, which means more bail/longer jail time.
Then the software guys complain and say they aren't racist, they are just applying the algorithm.
This article is trying to shut them up by saying their algorithm, in addition to being racist, doesn't work any better than simple common sense.
It is not an attack on the business model, just of the current state of the art.
excitingthingstodo.blogspot.com