When Sentencing Criminals, Should Judges Use Closed-Source Algorithms? (technologyreview.com)

Some judges in America have recently started using a closed-source algorithm that predicts how likely convicts are to commit another crime. Mosquito Bites shared an article by law professor Frank Pasquale raising concerns about the algorithms: They may seem scientific, an injection of computational rationality into a criminal justice system riddled with discrimination and inefficiency. However, they are troubling for several reasons: many are secretly computed; they deny due process and intelligible explanations to defendants; and they promote a crabbed and inhumane vision of the role of punishment in society...

When an algorithmic scoring process is kept secret, it is impossible to challenge key aspects of it. How is the algorithm weighting different data points, and why? Each of these inquiries is crucial to two core legal principles: due process, and the ability to meaningfully appeal an adverse decision... A secret risk assessment algorithm that offers a damning score is analogous to evidence offered by an anonymous expert, whom one cannot cross-examine... Humans are in charge of governments, and can demand explanations for decisions in natural language, not computer code. Failing to do so in the criminal context risks ceding inherently governmental and legal functions to an unaccountable computational elite.
This issue will grow more and more important, the law professor argues, since there's now proprietary analytics software that also predicts "the chances that any given person will be mentally ill, a bad employee, a failing student, a criminal, or a terrorist."

Comments

No

[Anonymous Coward]

No

Re: Short answer: No

[beelsebob]

There are many places where there simply is no open alternative, and the closed alternative was funded by a company that would never open its source, and shouldn't be forced to (after all, they paid for the research and development and should be able to profit from it)

No one is proposing forcing them to open source their code, they're simply proposing not using their code, and letting the market do the rest.

Companies don't have the right to make a profit just because they spent money on research. They only get to make a profit if their product is actually something useful. In this case, the parent poster argues that it's not useful since you can't verify any of the determinations it makes.

What do they do now?

[Hognoxious]

Imagine a judge just picks a number out of the air based on his own experience, opinions and yes, prejudices.

That's using a closed source algorithm, except it runs on a wet carbon platform rather than silicon.

Re:What do they do now?

[mhkohne]

Yes, and it's trivially easy to get people to believe the judge might have made a 'bad call' in your case because we all know that people make mistakes.

The problem with the black box algorithm is that in spite of years of things blowing up in their faces, a number that came out of a computer is still imbued with an air of 'rightness' that it most assuredly doesn't deserve. But that air of 'rightness' will make it much harder for a defendant to challenge inappropriately harsh sentencing because people think the computer must know something.

Re: Short answer: No

[bsolar]

Not using their code means an immediate loss of good tools/software, which will make it harder for government to fairly evaluate criminal cases. Court cases will become even more of what one person said versus what the other said, and therefore have worse outcomes than if technology was used to provide evidence.

This assumes that the tools/software are good (proof?), than it allows for easier and fair evaluation of cases (proof?) and that not using these tools would lead to worse outcomes for the defendant (proof?). To prove such tings would require to know and understand the details of how the algorithm work, which is impossible with the algorithm being closed.

Yes, of course there is potential for abuse in closed source software, but most likely there isn't blatant abuse, because that would look really bad if it got out that the government was, for example, targeting minorities or gays directly

It wouldn't be the first time the government got caught doing something reprehensible which assumed would never get public. Furthermore the "abuse" doesn't need to be malicious, it can also arise from unexpected behaviour of the algorithm in specific cases.

Having it closed source isn't ideal, but if it can be reliably shown to make accurate predictions, it's better than nothing.

No matter how accurate it is, the defendant has the right to know the reasoning leading to his sentencing and "the algorithm we trust said so" is definitely not enough.

Re:A better alternative

[Anonymous Coward]

> the US has a much more diverse population, and hence we have a much larger number of people who are difficult to rehabilitate.

The elephant in the room is: the US has a revenge justice. Not alone in this, the element of revenge is, alas, quite widespread in our societies, sometimes stronger, sometimes weaker.

In a modern society, revenge has no place. Protect the society, OK. Learn from mistakes (the "perpetrator", but also the society surrounding him/her), definitely. But revenge? We are at one level with IS?

Justic should be about getting to grips with what happened, why it happened and what to do to reduce chances that it'll happen again. While respecting the human rights of all involved.

And, as far as Breivik is concerned: kudos to the Norvegians. This guy has done something horrible (by all our standards), but he's being deprived of his liberty (you might argue that seems a necessary evil and I might concur). So society takes up a responsibility towards him, and Norvegian society is standing up to that, all the populist voices notwithstanding. Good.

You thing hanging him by his thumbs would be better? We had that in the Middle Ages, glad it's over.