Slashdot Mirror
When Sentencing Criminals, Should Judges Use Closed-Source Algorithms? (technologyreview.com)
Some judges in America have recently started using a closed-source algorithm that predicts how likely convicts are to commit another crime. Mosquito Bites shared an article by law professor Frank Pasquale raising concerns about the algorithms:
They may seem scientific, an injection of computational rationality into a criminal justice system riddled with discrimination and inefficiency. However, they are troubling for several reasons: many are secretly computed; they deny due process and intelligible explanations to defendants; and they promote a crabbed and inhumane vision of the role of punishment in society...
When an algorithmic scoring process is kept secret, it is impossible to challenge key aspects of it. How is the algorithm weighting different data points, and why? Each of these inquiries is crucial to two core legal principles: due process, and the ability to meaningfully appeal an adverse decision... A secret risk assessment algorithm that offers a damning score is analogous to evidence offered by an anonymous expert, whom one cannot cross-examine... Humans are in charge of governments, and can demand explanations for decisions in natural language, not computer code. Failing to do so in the criminal context risks ceding inherently governmental and legal functions to an unaccountable computational elite.
This issue will grow more and more important, the law professor argues, since there's now proprietary analytics software that also predicts "the chances that any given person will be mentally ill, a bad employee, a failing student, a criminal, or a terrorist."
When an algorithmic scoring process is kept secret, it is impossible to challenge key aspects of it. How is the algorithm weighting different data points, and why? Each of these inquiries is crucial to two core legal principles: due process, and the ability to meaningfully appeal an adverse decision... A secret risk assessment algorithm that offers a damning score is analogous to evidence offered by an anonymous expert, whom one cannot cross-examine... Humans are in charge of governments, and can demand explanations for decisions in natural language, not computer code. Failing to do so in the criminal context risks ceding inherently governmental and legal functions to an unaccountable computational elite.
This issue will grow more and more important, the law professor argues, since there's now proprietary analytics software that also predicts "the chances that any given person will be mentally ill, a bad employee, a failing student, a criminal, or a terrorist."
Government, as being tax funded, should use entirely open source software and open formats. Anything otherwise is favoring certain corporations (Microsoft formats for example) or having potential to be abused (FBI backdoors in government software).
We the people elected them. We the people should be able to inspect them.
Imagine a judge just picks a number out of the air based on his own experience, opinions and yes, prejudices.
That's using a closed source algorithm, except it runs on a wet carbon platform rather than silicon.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
Not using their code means an immediate loss of good tools/software, which will make it harder for government to fairly evaluate criminal cases. Court cases will become even more of what one person said versus what the other said, and therefore have worse outcomes than if technology was used to provide evidence.
This assumes that the tools/software are good (proof?), than it allows for easier and fair evaluation of cases (proof?) and that not using these tools would lead to worse outcomes for the defendant (proof?). To prove such tings would require to know and understand the details of how the algorithm work, which is impossible with the algorithm being closed.
Yes, of course there is potential for abuse in closed source software, but most likely there isn't blatant abuse, because that would look really bad if it got out that the government was, for example, targeting minorities or gays directly
It wouldn't be the first time the government got caught doing something reprehensible which assumed would never get public. Furthermore the "abuse" doesn't need to be malicious, it can also arise from unexpected behaviour of the algorithm in specific cases.
Having it closed source isn't ideal, but if it can be reliably shown to make accurate predictions, it's better than nothing.
No matter how accurate it is, the defendant has the right to know the reasoning leading to his sentencing and "the algorithm we trust said so" is definitely not enough.
Absolutely it should. Conincidentally, I legally changed my name to 1;drop table.
> the US has a much more diverse population, and hence we have a much larger number of people who are difficult to rehabilitate.
The elephant in the room is: the US has a revenge justice. Not alone in this, the element of revenge is, alas, quite widespread in our societies, sometimes stronger, sometimes weaker.
In a modern society, revenge has no place. Protect the society, OK. Learn from mistakes (the "perpetrator", but also the society surrounding him/her), definitely. But revenge? We are at one level with IS?
Justic should be about getting to grips with what happened, why it happened and what to do to reduce chances that it'll happen again. While respecting the human rights of all involved.
And, as far as Breivik is concerned: kudos to the Norvegians. This guy has done something horrible (by all our standards), but he's being deprived of his liberty (you might argue that seems a necessary evil and I might concur). So society takes up a responsibility towards him, and Norvegian society is standing up to that, all the populist voices notwithstanding. Good.
You thing hanging him by his thumbs would be better? We had that in the Middle Ages, glad it's over.
As a lawyer, I will admit most lawyers are not very mathematically savvy, except when it comes to calculating legal fees, then they turn into Srinivasa Ramanujam.
Frank Pasquale left out a couple of details in his opinion piece. First, these algorithms are only used in determining sentences, not to determine guilt. At that point in the trial guilt has been determined beyond a reasonable doubt. At the sentencing phase rules of evidence do not apply anymore and almost anything goes. That's why the prosecutor puts crying victims on the stand. There have been two supreme court cases (Malenchik vs Indiana and Loomis v. Wisconsin) that challenged the use of algorithms in sentencing and both upheld (in the later with some minor restrictions) that these algorithms can be used in sentencing. The conditions in general are that the algorithm has to be scientifically sound. That was the case in both cases that were challenged as there is existing peer-reviewed literature that examined the algorithms. Frank neglected to mention that. The secrecy of the algorithms is a consequence of patent and copyright law btw. The algorithms in these cases are a scoring function. Math is not (and should not) patentable or copyrightable. In this case the consequence is that the manufacturers only recourse is to keep it a trade secret. That could be solved better, but in my opinion people shouldn't get their hopes up that there's some exploitable loophole in the algorithm or something.
We can debate whether assessments (actuarial prediction instruments) should be used in sentencing or criminal justice. I’m very much in favor as it does reduce bias and leads to reproducible results. It’s much easier to control for biases in decision making with statistical methods than it is to control or fix bias in humans. Does anybody believe that human judgement is less biased? You can read up on the work of Paul Meehl who spend his lifetime showing that even simple assessment tests outperform the judgment of trained clinicians. Part of the sentencing is taking into consideration how likely the perpetrator is to commit a new offense. Humans suck at making predictions and estimating probabilities. This is no different in criminal justice.
Let me end this with pointing out some of the positive change that systems like this have brought: early release from incarceration. Low risk prisoners are more frequently released early (not just from overcrowded California prisons when ordered to do so by a federal judge), and then put on probation/parole. And work out well it did: http://time.com/4065359/califo... The expected crime wave from federally mandated early release didn't materialize. In my opinion thanks to these prediction models.
There are many things wrong with policing and criminal justice in the US, but the move to what’s generally referred to as “evidence based practices” (incl. actuarial prediction instruments) has been pretty positive. The great part is that both Dems and Reps are behind the idea of risk assessments so we might actually see some change for the better.