Slashdot Mirror
Software 'No More Accurate Than Untrained Humans' At Predicting Recidivism (theguardian.com)
An anonymous reader quotes a report from The Guardian: The credibility of a computer program used for bail and sentencing decisions has been called into question after it was found to be no more accurate at predicting the risk of reoffending than people with no criminal justice experience provided with only the defendant's age, sex and criminal history. The algorithm, called Compas (Correctional Offender Management Profiling for Alternative Sanctions), is used throughout the U.S. to weigh up whether defendants awaiting trial or sentencing are at too much risk of reoffending to be released on bail. Since being developed in 1998, the tool is reported to have been used to assess more than one million defendants. But a new paper has cast doubt on whether the software's predictions are sufficiently accurate to justify its use in potentially life-changing decisions.
The academics used a database of more than 7,000 pretrial defendants from Broward County, Florida, which included individual demographic information, age, sex, criminal history and arrest record in the two year period following the Compas scoring. The online workers were given short descriptions that included a defendant's sex, age, and previous criminal history and asked whether they thought they would reoffend. Using far less information than Compas (seven variables versus 137), when the results were pooled the humans were accurate in 67% of cases, compared to the 65% accuracy of Compas. In a second analysis, the paper found that Compas's accuracy at predicting recidivism could also be matched using a simple calculation involving only an offender's age and the number of prior convictions.
The academics used a database of more than 7,000 pretrial defendants from Broward County, Florida, which included individual demographic information, age, sex, criminal history and arrest record in the two year period following the Compas scoring. The online workers were given short descriptions that included a defendant's sex, age, and previous criminal history and asked whether they thought they would reoffend. Using far less information than Compas (seven variables versus 137), when the results were pooled the humans were accurate in 67% of cases, compared to the 65% accuracy of Compas. In a second analysis, the paper found that Compas's accuracy at predicting recidivism could also be matched using a simple calculation involving only an offender's age and the number of prior convictions.
An untrained person isn't pure chance. Pure chance is rolling a die. Untrained person is common sense.
Your ad here. Ask me how!
It has been shown that COMPASS overestimates the recidivism of black people by a factor of about two, while it underestimates the recidivism of white people at about the same rate -- while at the same time not even including race in the list of variables.
So it will rather deny bail to a black person which never commits a crime again. But it will let a white person go free on bail who later will become a repeat offender. As the exact inner workings of COMPASS are regarded as business secret, there were some experiments to find out why it is so bad at estimating the recidivism rate of people, and it seems that it totally overweighs social factors (stable/unstable family background, unemployment rate, debts etc.pp.), because there are many of them in the list of factors it considers. On the other hand, there are not many variables for the type of crime committed, and thus it does constantly underestimates those in the total. It would thus grant bail to a sexual offender who comes from a stable family background with steady income, though the recidivism rate of those is 70%, but it is only a single factor weighing against the offender. On the other hand it would deny bail to a petty thief, who does not have a stable family life, is indebted, has only short periods of employment and moves often.
Basicly: COMPASS is biased against people in poverty.
Are they much more accurate? How much?