Software 'No More Accurate Than Untrained Humans' At Predicting Recidivism (theguardian.com)

An anonymous reader quotes a report from The Guardian: The credibility of a computer program used for bail and sentencing decisions has been called into question after it was found to be no more accurate at predicting the risk of reoffending than people with no criminal justice experience provided with only the defendant's age, sex and criminal history. The algorithm, called Compas (Correctional Offender Management Profiling for Alternative Sanctions), is used throughout the U.S. to weigh up whether defendants awaiting trial or sentencing are at too much risk of reoffending to be released on bail. Since being developed in 1998, the tool is reported to have been used to assess more than one million defendants. But a new paper has cast doubt on whether the software's predictions are sufficiently accurate to justify its use in potentially life-changing decisions.

The academics used a database of more than 7,000 pretrial defendants from Broward County, Florida, which included individual demographic information, age, sex, criminal history and arrest record in the two year period following the Compas scoring. The online workers were given short descriptions that included a defendant's sex, age, and previous criminal history and asked whether they thought they would reoffend. Using far less information than Compas (seven variables versus 137), when the results were pooled the humans were accurate in 67% of cases, compared to the 65% accuracy of Compas. In a second analysis, the paper found that Compas's accuracy at predicting recidivism could also be matched using a simple calculation involving only an offender's age and the number of prior convictions.

Comments

Try again with deep learning

[Anonymous Coward]

Would be interesting to try again with current machine learning techniques.

Re: Try again with deep learning

[Anonymous Coward]

Wow, that's a strong sarcasm with you today.

Re: Try again with deep learning

[Sique]

60% of all homicides are suicides. Thus I doubt your numbers.

When it comes to homicides, most of them (about 90%) are perpetrated by yourself, your close relatives (spouse, parents, children) or your acquaintances.

Re: Try again with deep learning

[Wootery]

60% of all homicides are suicides

most of them (about 90%) are perpetrated by yourself

Neither of which are relevant to the racial disparity question.

Looking at Wikipedia, 52% of the USA's murders (i.e. not including suicides) are committed by black murderers.

Our AC troll presumably thinks that this proves that black people are naturally violent, or some such nonsense. Nope. The USA is very far from a colour-blind society, so the figures aren't all that surprising. Black Americans are far more likely to have the misfortune of growing up around violent gangs, etc.

Steven Pinker spoke about exactly this recently. There's no need to deny seemingly awkward facts in order to be a good liberal. Really, the facts show that racial inequality is still high in the USA, which supports the liberal view.

Re: Try again with deep learning

[Anonymous Coward]

>Black Americans are far more likely to have the misfortune of growing up around violent gangs, etc.

Also, as has been seen with the enforcement of drug laws, black Americans are also more likely to be punished for drug possession than their white counterparts, even though use is roughly equal among both demographics. I wouldn't be surprised if this lopsided enforcement also applies to murder investigations.

Re: Try again with deep learning

[Anonymous Coward]

Really? 60% of homicides are suicides? That's impressive. Especially when one considers that the definition of a homicide is when one person kills another and the definition of suicide is when somebody kills themselves.

So what you're saying is 60% of the time somebody kills another person, they killed themselves.

Man, what a world we live in.

Re: Try again with deep learning

[Anonymous Coward]

Black men are about 7% of the population but commit 60% of all homicides.

7% of the population? Of the US I assume? Which is 320 million or so, or the Black Male population is about 22 million people. According to the CDC, there were about 16,000 homicides last year. 60% of that is 9600. That means unless each homicide involves over 2,000 Black Men(or even 1000, or 100), your statistic is rather dubious in its usefulness.

I suggest you go back to Elementary School.

Re: Try again with deep learning

[temcat]

Our AC troll presumably thinks that this proves that black people are naturally violent, or some such nonsense. Nope. The USA is very far from a colour-blind society, so the figures aren't all that surprising. Black Americans are far more likely to have the misfortune of growing up around violent gangs, etc.

Is it really important for a potential murder victim whether the potential perpetrator is "naturally" or "culturally" violent? Do courts accept that as a defense in a murder case?

The first sentence in the quote above looks awfully like an attempt at a strawman. But of course, I do agree with there being no need to deny seemingly awkward facts.

Re: Try again with deep learning

[Anonymous Coward]

You forgot that the poorer someone is, the harsher the sentencing will be. In most areas, a public defender will ask for a plea bargain, then not take the case unless the defendant agrees to this. If you have your own lawyer who knows where to punch holes in a case, even one that would be normally open and shut, the DA likely will drop the charges because they will be wasting time with a case that they know they will lose, as opposed to an indigent case where they can "serve Justice" and run roughshod over someone who cannot afford better.

Drug laws are similar. Poorer people tend to be doing drugs in places outside of their homes that are easily busted or searched. You will not get the popo demanding entry to a gated community unless there is definite evidence, while rubber-stamped warrants in project housing are commonplace.

It isn't about white or black. It is about money. In fact, a DA is going to hit a poor white person with a tougher sentence, just so they can say that they dish out "justice" fairly, all the while ensuring Corrections Corporation of America (er, CoreCivics) keeps their campaign coffers full.

Re: Try again with deep learning

[Anonymous Coward]

60% of all homicides are suicides. Thus I doubt your numbers.

Since 60% of all apples are not oranges, I doubt your intelligence.

Common fucking sense is all you need to discern the obvious difference between homicide and suicide.

Re: Try again with deep learning

[gnick]

Our AC troll presumably thinks that this proves that black people are naturally violent, or some such nonsense. Nope. The USA is very far from a colour-blind society, so the figures aren't all that surprising. Black Americans are far more likely to have the misfortune of growing up around violent gangs, etc.

The root causes are interesting and important to improving our society. They're not important when deciding the disposition of a prisoner. The only thing that matters in that case if if they'll re-offend, not why they'll re-offend. The likelihood of recidivism is biased by all kinds of "unfair" metrics like gender and race. I personally don't think metrics like race should be used in the determination, but it would be more accurate if they were.

Re: Try again with deep learning

[gnick]

So what you're saying is 60% of the time somebody kills another person, they killed themselves.

I agree and so do I. Now get out of my head; it's crowded enough.

Re: Try again with deep learning

[Wootery]

Is it really important for a potential murder victim whether the potential perpetrator is "naturally" or "culturally" violent?

What? Of course not.

Do courts accept that as a defense in a murder case?

What? Of course not.

Re: Try again with deep learning

[Wootery]

The only thing that matters in that case if if they'll re-offend, not why they'll re-offend.

I personally don't think metrics like race should be used in the determination, but it would be more accurate if they were.

You've contradicted yourself.

If you believe that racial data should be excluded from the calculation of how likely the convict is to re-offend, that means you value something else alongside whether they'll re-offend.

That's my position, too. Discarding racial data really does diminish the accuracy of the odds-of-re-offending calculation, but we should do it anyway.

Re: Try again with deep learning

[sheph]

In addition to having the misfortune of growing up around gangs there are other factors like attitude. When you grow up with a chip on your shoulder, being told you can't trust people of a particular color, discouraging assimilation into society, encouraging entitlement, etc. This all factors in as well. It's the reason they join gangs. They don't believe they are safe on their own and that there is strength and safety in numbers.

Re: Try again with deep learning

[gnick]

You've contradicted yourself.

I don't think I did. I said that including "unfair" metrics like race help get the most accurate prediction. They do. I didn't say that getting the most accurate prediction was the most important thing to do. I pointed out that the "why" is unimportant to the "if". This is true for any "why" that isn't included in the metrics. The "why" is interesting for other reasons.

Discarding racial data really does diminish the accuracy of the odds-of-re-offending calculation, but we should do it anyway.

Yes.

Re: Try again with deep learning

[snakeplissken]

| So what you're saying is 60% of the time somebody kills another person, they killed themselves.

I agree and so do I. Now get out of my head; it's crowded enough.

I'm spartacus! And so am I!

Re: Try again with deep learning

[Wootery]

the "why" is unimportant to the "if"

But that's just it - the 'why' does matter; we're discussing deliberately discarding certain 'whys' from our estimator, in the name of not being racist.

If the reason a convict is denied parole is because he's black (i.e. were he not black then his estimated odds of re-offending would have been below the threshold), then the 'if' behind his denial surely matters.

Re: Try again with deep learning

[Anonymous Coward]

He didn't say 60% of black men, moron

Re: Try again with deep learning

[gnick]

You left out:

This is true for any "why" that isn't included in the metrics.

Being black is a useful metric for making an accurate prediction. We agree on that and agree that it should be ignored. The social inequality that causes blacks to have higher recidivism rates is interesting, but not part of the equation. The fact that blacks are searched more often is interesting, but we don't factor that in either. What's important when guessing whether a person will re-offend is whether he's black, not why blacks are more likely to re-offend. The reasons why black offender have higher recidivism rates are interesting and important, but the only thing important to factor into the prediction is if he's black.

Re: Try again with deep learning

[ewibble]

Some reasons are important. For example if the crime is say drug possession, and black people are searched more, then the black person is more likely to be caught even if both white and black people have exactly the same rate of redoing the crime. We only count residivism if the person is caught.

I assume the goal is to reduce the actual offending not the reduce the caught offenders.

We could reduce residivism to 0 instantly if just got rid of all law enforcement. In case of drug possession maybe not a bad idea, assault not so much.

Re: Try again with deep learning

[Dragonslicer]

Is it really important for a potential murder victim whether the potential perpetrator is "naturally" or "culturally" violent?

No, but it may be important to a society that has a desire to reduce the murder rate.

Re: Try again with deep learning

[ceoyoyo]

Not in the US. In the US black men commit murder at a greater rate than other demographics, but their total is still less than the white male total.

Re: Try again with deep learning

[ceoyoyo]

Race is likely a surrogate for other factors, as the OP pointed out. Yes, putting it in your model may make your model more accurate, but it's demonstrably not a great surrogate (it doesn't perfectly represent the actual causative factors). So you might improve your model, but you'll improve it MORE by using the proper variables.

It's even more dangerous when you apply group level surrogates to individuals. It's quite possible that part of the reason the random internetters are outperforming the computer model is that they're using surrogates like race in a more intelligent way. Or simply that their background knowledge is more up to date: I expect that the relationship of race to crime has changed quite a bit since the 80s.

Re:Try again with deep learning

[K. S. Kyosuke]

How do your proposed techniques deal with explaining their reasoning process? Or is your proposed machine a black box that metes out justice that nobody can question, appeal, or understand?

Re: Try again with deep learning

[ThePawArmy]

When it comes to homicides, most of them (about 90%) are perpetrated by yourself, your close relatives (spouse, parents, children) or your acquaintances.

Seems like we should do "something" about this person and their spouse, parents, children and acquaintances.

Re: Try again with deep learning

[temcat]

That I agree with.

Re: Try again with deep learning

[Anonymous Coward]

That's probably why I didn't mention 60% of black men, but 60% of the total homicide figure.

60% of 16000 is 9600.
60% of 22000000 is a number I didn't reference.

Re: Try again with deep learning

[Wootery]

In addition to having the misfortune of growing up around gangs there are other factors

Yes, of course. One could do a PhD on those other factors. The 'etc' I used covers all sorts of things.

Re: Try again with deep learning

[Wootery]

Either I'm misreading you, or you're not making sense.

We're using the word 'why' to refer to the dimensions of the machine-learning data-set, right? So 'blackness' is a dimension, and is a 'why' (i.e. it's something the machine can factor into its decision to permit/deny parole).

Obviously the machine learning machine isn't going to comprehend the nuance of societal racism. That isn't the point. The point is that we're agreed we shouldn't input race data into the system.

the only thing important to factor into the prediction is if he's black.

But we just agreed that we shouldn't factor in whether the convict is black. Are you taking that back?

Re: Try again with deep learning

[Wootery]

I assume the goal is to reduce the actual offending not the reduce the caught offenders.

A sensible distinction, but there's more to it: we don't want to implement a racist system even if it's more effective at reducing offending rates.

In other words, we need to treat political correctness as an engineering goal.

Re: Try again with deep learning

[Anonymous Coward]

the only thing important to factor into the prediction is if he's black.

But we just agreed that we shouldn't factor in whether the convict is black. Are you taking that back?

I'm repeating myself, so AC. If your goal is to get the most accurate prediction, you leave the race metric in. Getting the most accurate prediction isn't the only goal. Race should not be factored in even though it would improve the accuracy of the prediction.

Of course, no blockchain tech here

[Anonymous Coward]

so what would you expect?
Add some blockchain goodies and everything will work perfectly....or at least the next round of funding...

No bad software

[hcs_$reboot]

Only bad programmers/designers.

Re:No bad software

[Anonymous Coward]

Excellent work

Re:No bad software

[Anonymous Coward]

Good middle class programmers would never unintentionally reinforce the pre-existing class system by allowing bias into their datasets or algorithms.

Re:No bad software

[hey!]

Well, there's bad (i.e., stupid) clients too. They're responsible for a lot of bad software.

If a customer wants to buy magic software without an understanding of what it does or proof that it even works, what are the programmers supposed to do about that? They just report to work and build what their boss tells them to build, and he tells them to build what the customer will buy.

Re:No bad software

[Anonymous Coward]

Only bad programmers/designers.

With 137 variables, there must have been one hell of a monumental fuck-up on the programming/design side if that software could be re-designed to actually hold value.

Either that, or it's not just bad software. It's a bad idea.

Re:No bad software

[rhazz]

Deciding which data points are predictors of a certain outcome in regards to crime has nothing to do with software, programmers, or designers. That's the job of statisticians, who probably weren't involved in the creation of this software.

Using monkeys maybe?

[Anonymous Coward]

They are better at investing so how about giving them the chance to predict recidivism with some banana's?

Question

[Kokuyo]

It seems obvious that someone with more relapses in the past will also be more likely to do it again. However, I will assume that at that point, a judge wont allow for bail anyway so if this is about people with three or less offenses on their record, I'd imagine that ONLY going by the criminal history is going to be inaccurate no matter who or what is looking at it.

Isn't this more a case of bad data as opposed to bad programming? Because "no more accurate than an untrained person" implies pure chance.

Re:Question

[Actually, I do RTFA]

An untrained person isn't pure chance. Pure chance is rolling a die. Untrained person is common sense.

Re:Question

[AmiMoJo]

It seems obvious that someone with more relapses in the past will also be more likely to do it again.

Unless those relapses caused them to enter a programme or get additional support to avoid relapsing again. Or maybe their personal circumstances changed. It's very difficult to write software that can evaluate an individual's personal circumstances.

Re:Question

[gweihir]

Since common sense is very rare, and untrained person is likely just a feeling ob being more or less threatened, with no connection to the actual threat.

Re:Question

[Barsteward]

If humans are no better than a machine then how can a machine programmed by humans do any better

Re:Question

[Anonymous Coward]

Untrained person will introduce religious and racial bias. The software might actually be better (maybe more consistent, more fair.)

Re:Question

[mjwx]

An untrained person isn't pure chance. Pure chance is rolling a die. Untrained person is common sense.

That would be a good point if common sense were common.

For every person who applies logic and reason to a case (I.E. shows remorse, now has steady job, impetus for theft no longer present) there are 2 or 3 people who apply batshit insane rules (I.E. He's Ginger, so he'll steal again).

Re:Question

[Anonymous Coward]

If humans are no better than a machine then how can a machine programmed by humans do any better

Perhaps it can't. Sometimes the best decision, is to throw the bad idea away.

Unfortunately, there's too much money in state and government coffers to ignore, so greed and corruption continue to be fed, no matter how pointless and worthless a bad solution is.

Re:Question

[omnichad]

On the other hand, this software is nothing but bias. Free will vs. determinism and they're siding with determinism. It might match the religious and racial bias of an average human.

Re:Question

[Anonymous Coward]

Your concept of free will is a comforting lie. Nothing more.

We can see the neurons light up on fMRI telling us what your decision will be---and this consistently happens before you "decide" what to do.

Whatever the ultimate truth is, it is closer to determinism than the Western concept of free will.

Re:Question

[EndlessNameless]

If humans are no better than a machine then how can a machine programmed by humans do any better

This is a stupid, Luddite sentiment. Computers are already far better than humans at a wide range of tasks. This increases dramatically if your usage of "machine" include robots performing physical tasks.

Deep learning is opening the doors to new machine skills. Quantum computing will likely open a few more in the next few decades. It's anyone's guess what comes after that.

Humans were smart enough to make a machine that could do some things better than they could. We used those new capabilities to make better machines---again and again. The only thing that stops this process is the possibility that a final generation of machines will be incapable of developing meaningful advances for a new generation.

Re: Question

[Anonymous Coward]

Actually, the fact that neurons light up before your are conscious of a decision you've made before neurons light up doesn't mean free will doesn't exist.

One wouldn't expect an instantaneous link between physical phenomena in a world where physical laws limiting information transmission speeds exist, for instance. Additionally, the underlying physical phenomenon of consciousness isn't even understood at anything close to a high level, so, once again, meaningless to try to say anything about it with that information.

Re:Question

[blindseer]

One measure of intelligence is the speed at which one can make a decision. A very basic intelligence test is the speed at which a person can match patterns. A test like this run over perhaps 15 minutes will get a very high correlation to a pen and paper test run over an hour or two where one must perform mathematics problems or word comprehension.

So, if speed of making a decision is defined as intelligence then we can define the criteria for a decision, put that in a way a machine can compute quickly, then we know have a machine that is highly intelligent on the one task it was programmed to perform.

We can train people in this algorithm, give them a stack of papers to sort through in making this decision, and get the same results as the computer. But the computer will do this same decision process with greater speed and accuracy.

Re:Question

[gweihir]

Bullshit. You have fallen for junk-science. There is no way to know when a person actually decides things and hence there is no reference point. The whole experimental set-up is fundamentally flawed.

Re:Question

[ewibble]

People are actually quite good at making judgements of people based on very little information, https://www.webmd.com/balance/..., it makes sense really, in the real world we have very little actual information about someone.

Re:Question

[ceoyoyo]

"There is no way to know when a person actually decides things"

You called bullshit on someone citing actual research with an unsupported, absolute statement.

Re:Question

[ceoyoyo]

Yes. And that process does better than what's actually being used.

Scary.

It would be interesting to see how actual experts perform.

Re:Question

[james_gnz]

Since common sense is very rare...

Common sense itself is common by definition. You might not think so, given all the things that people claim are common sense. However it is rare that an appeal to common sense refers to something that is actually common sense, because if that thing actually had been common sense, it would have been unlikely for a disagreement to have arisen around it, so there would have been no cause for making an appeal to common sense.

Obvious really. It's just common sense.

Re:Question

[CanHasDIY]

It doesn't count as "actual research" unless they post the actual research.

"I agree with this" != "this is factually correct"

Re:Question

[CanHasDIY]

If humans are no better than a machine then how can a machine programmed by humans do any better

This is a stupid, Luddite sentiment.

Only if you're the sort who immediately gets butt hurt and defensive when you see something you assume offends you.

A person who isn't that sort might look at the question with a greater philosophical viewpoint, e.g. "if the man who wrote the program is flawed, wouldn't it follow suit that the program may be flawed as well, in the same fashion?"

Gedankenexperiments tend to be wasted on plebs and the easily offended.

Re:Question

[ceoyoyo]

Sure it does. If you ask nicely, I'm sure someone would provide you a link. Or you could google it. I think it was even discussed on Slashdot.

He also described the results. The GGP's response was "that's impossible."

"I agree with this" != "this is factually correct"

Not sure how that's relevant, but I agree!

Ha!

[Anonymous Coward]

They seem to be missing a crucial piece of demographic information. Put it back in, and I bet both humans and machines can get 90%+ accuracy.

Re: Ha!

[Anonymous Coward]

Sorry to burst your white supremacist bubble but doing that would be unconstitutional. Not everything is explainable by race.

Re:Ha!

[Sique]

In a certain way you are right, but you got it backwards.

It has been shown that COMPASS overestimates the recidivism of black people by a factor of about two, while it underestimates the recidivism of white people at about the same rate -- while at the same time not even including race in the list of variables.

So it will rather deny bail to a black person which never commits a crime again. But it will let a white person go free on bail who later will become a repeat offender. As the exact inner workings of COMPASS are regarded as business secret, there were some experiments to find out why it is so bad at estimating the recidivism rate of people, and it seems that it totally overweighs social factors (stable/unstable family background, unemployment rate, debts etc.pp.), because there are many of them in the list of factors it considers. On the other hand, there are not many variables for the type of crime committed, and thus it does constantly underestimates those in the total. It would thus grant bail to a sexual offender who comes from a stable family background with steady income, though the recidivism rate of those is 70%, but it is only a single factor weighing against the offender. On the other hand it would deny bail to a petty thief, who does not have a stable family life, is indebted, has only short periods of employment and moves often.

Basicly: COMPASS is biased against people in poverty.

Re: Ha!

[Nidi62]

To be fair, your last example gives a lot of evidence that they could be a flight risk, which is an important reason to deny bail. Really, both of your examples should probably be denied bail.

Re:Ha!

[Anonymous Coward]

As the exact inner workings of COMPASS are regarded as business secret

I don't understand why this one fact, alone, isn't enough to completely disqualify the program from participating in legal decisions.

Re: Ha!

[Anonymous Coward]

Flight risk for petty theft? The punishment for petty theft vs. the hardship of being a fugitive on the run makes me think you didn't put much thought into your post.

Re:Ha!

[CanHasDIY]

Basicly: COMPASS is biased against people in poverty.

Well then, it sounds like it's functioning as designed, just like the legal system in its current state.

Re: Ha!

[another_twilight]

The hardship you mention is felt to a greater extent by someone who has a life and identity that they would lose or lose access to and who would be forced into a future where they would have little stability, no contact with family, would probably not be able to hold down decent jobs or even poorer jobs for any length of time and may end up resorting to petty crime to get by.

Given that the person in the GPs example is already living that kind of life, I think you are overestimating the disincentive that being a fugitive might be.

A person with a stable life, supportive family and/or friends is likely to be able to rebuild/recover after a conviction. Someone living more marginally is less likely to be able to do so, so the decision to stay or flee isn't evenly weighted.

That sort of life doesn't lend itself to teaching someone the value of long term planning nor provide much opportunity to practice the sort of reasoning needed to see that copping the conviction is probably a better outcome than fleeing prosecution.

So while the choice may be simple and clear for you, I don't think _you've_ made much effort to model the position of the person the GP described.

Parameters

[Anonymous Coward]

A big part of risk of reoffending lies in the person's relationships in their life after prison. If they are reconnected to normal society they will be less likely to reoffend than if they lack connections or connect with cother criminals.

Obvious Mistake

[freeze128]

The programmers spent all their time on writing the acronym for the name of the software, and not enough time actually making the software work.

"Headlines no more accurate than stupid clickbait"

[locater16]

Tl;Dr Single old program tested in situation vendor says is inaccurate use of software, software doesn't work well. Thus all programs will forever be terrible at this task and these computer guys should give up and do something useful. Like writing headlines for news sites!

Or bad metrics for your model

[Togden]

Isn't this precisely what you would expect when the information gathered to make the decision isn't influential enough on the outcome. It says they have 137 variables, which were as useful as 2. It suggests that the additional variables are either unrelated to the outcome, or are strongly related to the 2 suggested such that either way they provide no additional accuracy.

Re: Or bad metrics for your model

[Anonymous Coward]

But 137 sounds better than 2 for marketing.

Re:Or bad metrics for your model

[EndlessNameless]

It suggests that the additional variables are either unrelated to the outcome

Or their application was slapped together haphazardly in order to start bringing in those contract dollars as soon as possible. The other 135 variables could provide useful information, if their analysis of that information were better. Until this study, there was no incentive to perform well because the cost of poor performance is borne by convicts.

I am far more willing to believe that crapware is being shoveled into government computers at taxpayer expense.

Bear in mind that this system performs as well as untrained humans. If humans with correctional or psychological expertise perform better than untrained humans (which I assume is true), then this system is embarrassingly bad. It's doing a worse job than the people it's supposed to help.

Re: Or bad metrics for your model

[sjames]

It's a classic flim flam technique. Take a normally simple transaction and weave a complicated story around it with many steps and complicated procedures. Make sure the story involves many (fictional) people with their own idiosyncrasies. Keep them focused on the seeming logic of each individual step so they won't look at the overall situation and see the scam coming.

Re:Or bad metrics for your model

[blindseer]

Bear in mind that this system performs as well as untrained humans. If humans with correctional or psychological expertise perform better than untrained humans (which I assume is true), then this system is embarrassingly bad. It's doing a worse job than the people it's supposed to help.

With this software they'd need a handful of trained people to author the algorithm and then many more lesser trained people can apply this software without introducing personal bias. It's also a kind of double blind testing. The people writing the algorithm don't see the offenders, and the people applying the algorithm don't know the algorithm to introduce a bias to it. This is presumably more fair as personal biases cannot be applied to individual offenders. If someone involved in creating the algorithm has some kind of bias then this will show in the code and others looking at the code can likely see this and correct it in the development process.

This should keep things consistent and improve the rate that offenders can be processed. As people are not machines this algorithm cannot take into account things it does not know, so even with 137 items of data to deal with there will be false positives and false negatives.

Is this process "fair"? Nothing is fair. The parole board cannot take into account things they do not know, and they cannot know the "soul" of the offenders they may be releasing. If age and prior offenses is all they need to be as accurate as this complex algorithm then they need to compute this on a chart, give the trained people some discretion on the decision, within some range, and that's likely to be as "fair" and consistent as any computer. If the person doing the decision is not trained on how to apply this "X factor" that they have control over then they can simply rely on the pre-computed chart that's based on age and prior record.

Duh, missing other data dimensions

[Anonymous Coward]

So it tried to be blind and fair and couldn't find a useful pattern. Now add race, location, education, and see what happens.

Re:Duh, missing other data dimensions

[hey!]

It tried to be fair and actually failed, because it uses a methodology that clearly wasn't designed by a statistician.

The program uses over a hundred factors in its classification scheme, but statisticians and data scientists make a point of pruning factors because long experience has shown that introducing many irrelevant factors actually reduces predictive accuracy. And just because race is not an explicit factor doesn't mean that the algorithm is race blind either. It's entirely feasible to given the huge number of factors involve to recover the subject's race with a better-than-chance reliabilty, whether explicitly or implicitly; intentionally or even by accident.

Now the program's score is equally correlated with reoffending rates whether the subject happens to be white or black, which sounds impressive and color-blind -- to a layman. To a mathematician not so much. It's actually quite easy to produce this result by tweaking your model, implicitly recovering race in the manner suggested above and forcing it to produce a result that looks right -- in aggregate.

But what a statistician wants to know is about conditional probabilities, and it turns out that when applied to retrospective data the program is twice as likely to commit a type 1 error (falsely predicting reoffending) for black subjects as white. If this makes the whole process of achieving fairness sound hard, that's because it is. Color-blindness in aggregate isn't the same as color blindness on a case-by-case basis, and that's the thing that actually matters.

Ultimately you want criminal justice decisions to be based on reason, and mathematics is the purest form of reason there is. And because you want those decisions to be based on reason, they have to be transparent. Secret methods for arriving at decision-making are fundamentally antithetical to our concept of justice.

Re: More data does not always mean more accurate

[Anonymous Coward]

Well if you spend a couple decades arresting a significant portion of the adult male population of a certain group, and then don't give their schools funding because the higher funded schools are in suburbs, it isn't a big surprise when the kids grow up to have a low IQ or are criminals.

Has nothing to do with their ethnicity other than people of certain ethnicities live in certain areas and smoke certain plants which are different than the other plants the other ethnicities smoked.

Re: More data does not always mean more accurate

[Anonymous Coward]

Studies will show that IQ correlates highly with genetics. Some studies will show IQ is 50% genetic, others will show it's 90% genetic. If criminal behavior is highly correlated to IQ, and it seems that it is, then better funded schools will not significantly change the criminal tendencies of a population.

People will point to the Flynn Effect to claim that IQ is highly correlated to education. Here's an interesting fact, the same Dr. Flynn that found this effect will admit to a genetic component to intelligence. He sought funding to investigate the genetic influence on intelligence but no one would do so. He has said this is an admission that genetics plays a large role in intelligence, and people know this, but no one wants to be the person that funds the study to prove it as that might be career ending. If people discovered beyond a doubt that the genetic background that correlate to skin color also correlate to intelligence then... I don't know what would happen but it's not likely to be good.

Maybe there's no better algorithm

[istartedi]

So. Is there any better algorithm? You'd think that if there were a consensus among people studying this, they'd code in the consensus. Maybe the interesting thing here is that age and priors are the only useful information for predicting recidivism. This doesn't seem like rocket science. We've got decades of data. We ought to be able to run some other algorithms over it--something that takes into account a 3rd variable, and see if it helps. Maybe it does. Maybe it doesn't.

Re:Maybe there's no better algorithm

[AmiMoJo]

They are trying to solve the wrong problem. Rather than trying to quantify people, the solution to people reoffending is to provide better support to everyone. Stop wasting money on software and start investing in programmes that help reform offenders.

Reform programmes are also a much better way to evaluate people, because their progress in the programme is much easier to measure and requires them to meet goals that change their behaviour and future life chances. That's why sensible systems hand out a sentence which can then be reduced through participation and good behaviour.

Re:Maybe there's no better algorithm

[Anonymous Coward]

Reform programmes are also a much better way to evaluate people, because their progress in the programme is much easier to measure and requires them to meet goals that change their behaviour and future life chances.

But reform programs aren't seen as "tough on crime", thus politically nonviable.

Re:Maybe there's no better algorithm

[Anonymous Coward]

The problem in the current political climate is, such algorithms will always be hamstrung by political correctness. A large part of the article already goes on about racial fairness because reflecting reality will inevitably lead to "racist" algorithms, and "fair" algorithms will perform poorly.

Re:Maybe there's no better algorithm

[Anonymous Coward]

My problem with "tough on crime" is what that tends to actually translate to. The population hears "tough on crime" and thinks "they're going to have patrols and if my house gets broken in to, they're going to put all the resources they can in to finding who did it".

When the politicians say "tough on crime" what they mean is "we're gonna set up speed cameras and hide police on sidewalks in front of walgreens (actually happened in front of my house two days ago, a giant fricken SUV, thanks for destroying the sidewalks in my neighborhood police) and catch every speeder we can to increase revenue. Also, god help you if you have a joint".

Tough on crime will never mean what the population thinks because that's expensive, dangerous and hard to do. It'll just continue going after petty crimes that almost nobody cares about, because low hanging fruit and technically doing what they claimed they'd do.

Re:Maybe there's no better algorithm

[Gilgaron]

It's relatively easy to compare the algorithm to untrained people and confirm it isn't any better. It is harder to make a better algorithm, because the database may not even contain all the variables that really are important to gather, nor may gathering the best ones even be possible. (e.g. if the truly best variable was "positive relationship with grandfather", this information has likely not been collected nor collated; variables that seem important may merely be correlated with actual variables, e.g. "likes fruits and vegetables" versus "stable home life with family meals" or whatever). Even experts that are better than the algorithm may not realize what factors they're unconsciously taking into account.

Re:Maybe there's no better algorithm

[Ol Olsoc]

They are trying to solve the wrong problem. Rather than trying to quantify people, the solution to people reoffending is to provide better support to everyone. Stop wasting money on software and start investing in programmes that help reform offenders.

Reform programmes are also a much better way to evaluate people, because their progress in the programme is much easier to measure and requires them to meet goals that change their behaviour and future life chances. That's why sensible systems hand out a sentence which can then be reduced through participation and good behaviour.

They are trying to solve the wrong problem. Rather than trying to quantify people, the solution to people reoffending is to provide better support to everyone. Stop wasting money on software and start investing in programmes that help reform offenders.

Support does work. California's Folsom Prison is an example. At one point it had low recidivism numbers, and a lot of prisoner support. The we got "tough on crime" overcrowded the place, eliminated help for the prisoners, and now the place is a holding area between crimes.

Reform programmes are also a much better way to evaluate people, because their progress in the programme is much easier to measure and requires them to meet goals that change their behaviour and future life chances.

There are some more physical factors that are obvious. Age is one. Given that most prisoners are men, testosterone level will lower as they age. But I think there is one issue that is really important - impulse control.

Impulse control is one aspect of a person's personality that can be observed and improved in most cases. It is also involved in crimes of opportunity.

The impulse control or lack of it is probably involved in the second selection criteria of criminal record.

But the concept of treating prisoners well and providing education and needed treatment does work.

Re:Maybe there's no better algorithm

[Anonymous Coward]

The USA justice system has no interest in reforming the guilty, they are only interested in punishing the guilty. It is a fundamental difference between the USA system and systems in many other western countries.

Re:Maybe there's no better algorithm

[AHuxley]

Does the citizen have a high school education?
Is the citizen married?
Did the citizen have a job?
The decades of data shows who is doing crime, what part of the USA and who the victims are.
The data sets exist but the virtue signalling politics and optics around them are not good for inner city party politics.
Political correctness is holding the correct use of the inner city crime data sets back.
Decades of huge new amounts of education support from the gov, private sector did not change parts of the USA.
The crime rates in certain areas are getting very bad very generation. Education, more health care, income assistance and support for generations did not help.

Local police with federal support need support to find the criminals in a few parts of the USA given crime data not politically correct ways to hide the data sets.

Re:Maybe there's no better algorithm

[Anonymous Coward]

Your argument is solid from a philosophical standpoint but this program doesn't do what you think it does. This program is utilitarian. All it does it set the amount of bail for the current convict standing before the judge. It is not tasked nor could be made to "provide better support to everyone." You're asking a toaster to solve immigrant wheat farmer's labor problems. The toaster just toasts bread, man.

Re: More data does not always mean more accurate

[Anonymous Coward]

Keep talking out of your behind (without facts)

Re:More data does not always mean more accurate

[quantaman]

They said the software used 137 data points on determining the probability of re-offending but they were no better than if someone use just 2, age and prior convictions. Perhaps I've had more statistics training than most but this seems highly probable. This is pretty basic data analysis, or so I thought. If you take a bunch of data points and correlate them to re-offend rate there will be some data points that correlate more than others. If one doing the analysis tossed out the data points that had little to no correlation then the accuracy of the predictive value will still be effectively unchanged.

I don't think pure statistics is the proper approach as much as machine learning. To be honest the problem sounds like something that could be tackled in an undergraduate course, "here's your variables, there's your outcomes, run a classifier, and submit your results".

Perhaps there's something fundamentally difficult about getting above 2/3 accuracy, but it seems you should REALLY be able to beat untrained workers on a problem like this.

I suspect this is just a case of a product built in the late 90's on weak ML (or some homegrown stats) and they never felt the need to improve their results since.

Another thing that I've learned, and I'll admit is controversial to the SJWs out there, is the correlations between ethnicity and intelligence, and between criminal tendencies and intelligence. This is not controversial to the people that do this analysis, it's been established with considerable evidence.

There's no question that there's correlations between IQ and skin colour, the question is whether that's the characteristic of the ethnicity or race or due to socio-economic factors.

Those with an IQ around 85 or 90 (depending on who you ask) will be most likely to be criminals. Above that IQ there is greater profit in getting a job. Below that IQ the people will have problems concocting the means to break the law and still come out ahead.

The biggest predictor of criminality is age, the cause isn't poor earning potential, it's poor self-control and ability to anticipate consequences.

People from certain areas of the world will, on average, have a lower IQ. Average IQ, by definition, is 100.

If these people want a more accurate indication of criminal behavior then give an IQ test. They won't do that though because people with a certain ethnic background will "fail" this test and be considered more likely to offend.

They won't do that because you'd be denying people bail for being dumb. Anyway, you're probably not getting useful data because we're already dealing with convicted criminals and we have data on their criminal history.

With this trend of ethnic background having some correlation to skin color this algorithm would immediately be considered "racist" and be tossed out by the SJWs. Even though it would be highly accurate in determining future criminal behavior we can't tolerate a "racist" algorithm.

Why do we see more people with dark skin in prisons? Not because of some inherent racism. It's because low IQ people are more likely to break the law, and people with dark skin tend to have a lower IQ. This should not reflect on any individual because "trend" does not mean "will" or "did". Also, even with a 90 average IQ in a population still leaves a lot of room on a bell curve for many geniuses in that population.

Posted anonymously because I'm sure just mentioning these indisputable facts will likely get me labeled a racist.

If you wanted to discuss the role of race in a predictive algorithm that's valid, there's definitely an issue where machine learning algorithms can learn racial bias, even when they have to infer it through secondary measures. And depending on your view that's a good thing (it improves accuracy) or a bad thing (people are literally being judged by the colour of thei

Bad budgeting and management

[Anonymous Coward]

How could this have even been allowed to happen unnoticed?

Very obviously they let the same company that developed it test it. Most likely they either did not research said company very well or chose not to by going through an agency.

I would guess another of their mistakes was how they proposed a solution be created. Likely this is a set in stone windows compiled program which requires updates and maintenance via patches vs a simple web page. They probably wanted it fast and did not stipulate the usage of AI which led to the simplistic table system it apparently runs on. The low quality denotes also that this was done on a shoe string budget with little oversight.

A disaster of poor architecture, planning, oversight and execution.

Someone with some idea of how pattern matching works coupled with neural networks and modern multi user secure web architecture should probably be placed in charge of version 2.0

Doesn't matter

[nospam007]

The software works for free 24 hours a day, 7 times a week, doesn't need sick days, vacation, maternity leave nor does it want a pension when it ill be replaced by a much better AI version.

Re:Doesn't matter

[Anonymous Coward]

Nope. It's likely extremely expensive with subscription fees as well as a fee for each person it's used on. Along with that, they get to data mine the criminal records and resell the info. A spreadsheet/chart/lookup table would be the best and cheapest option paired with the gut feeling of the judge.

Re:Doesn't matter

[Anonymous Coward]

This is government we're talking about. Efficiency doesn't matter since governments don't go bankrupt, they raise taxes and print money. The primary concern of government is to employ as many people as possible and hand out retirement benefits far in excess compared to companies that are held accountable for their labors.

Re:Doesn't matter

[Anonymous Coward]

it does matter if it's your bail hearing, or that of someone who committed a crime against you.

In that case you do not want the equivalent of an untrained human. You want the equivalent of a trained human. Ideally even better;

Re:Doesn't matter

[Anonymous Coward]

Most western nations don't run their operations like that. Where do you live?

Re:Doesn't matter

[geekmux]

The software works for free 24 hours a day, 7 times a week, doesn't need sick days, vacation, maternity leave nor does it want a pension when it ill be replaced by a much better AI version.

137 variables within an algorithm has been running untouched for twenty years now. Had this study not taken place, it would have gone untouched for another twenty years, regardless of AI advancement, because everyone would be sitting back just assuming that it's doing a "good" job.

The most valuable question in the world is Why. Repeat it as many times as necessary until you get stupid people to stop saying "Because we've always done it this way."

Re:Doesn't matter

[crunchygranola]

137 variables within an algorithm has been running untouched for twenty years now. Had this study not taken place, it would have gone untouched for another twenty years, regardless of AI advancement, because everyone would be sitting back just assuming that it's doing a "good" job.

Even setting aside advances in machine learning, the assumption that the data sample used to train the algorithm would be equally valid 20 years later is stunningly foolish.

Any such tool should regularly be updated and revalidated to establish its performance.

I wonder how well this program was ever vetted and validated. Like the Liebold voting machines sold at about this same time that were proprietary pieces of security-hole ridden garbage, I suspect this product was sold simply through connections and the naive assumption that using a computer for something is automatically "better".

Sex discrimination

[Anonymous Coward]

They use sex as input for a sentencing decision? That cannot be legal.

Re:Sex discrimination

[Anonymous Coward]

Gives a good head - bad head metrics is collected during the sentence by the prison guards and sometimes also other prisoners. Speaking of the metrics, my most successful investment decisions have never based on historical performance of an individual stock but on the sales volumes and stock type compared to the state of the market on the whole. The question is how this herd intelligence could be used in the case of sentencing.

I work for a parole board

[Anonymous Coward]

We use race primarily as a recidivism indicator. We have one of the lowest recidivism rates among the 2870 prisons in the state of Alabama.

Software ?

[NormanHaga2580]

Pretty much anyone who has written and used artificial intelligence against a random time series can tell you that software is terrible at prediction of future outcome.

Why do I say this? I have been writing AI software for three years examining lottery prediction. Every model I have written is great at producing numbers, but Terrible at predicting the lottery, Maybe someday, but not today.

With software being so bad at predicting random events, why would someone attempt to use it to predict the random event of whether someone will reoffend.

It is not surprising that humans can use two variables and have an equal or greater chance of predicting whether a person will reoffend. It is pretty well understood by parole officers, police, and the courts that a person mellows with age and becomes less likely to commit a crime.

The comment that attempts to associate crime with intelligence made by Quantaman shows a serious misunderstanding of crime and society and intelligence.

Re:Software ?

[grub]

Lottery prediction is trying to predict random numbers, I would hope these are not predictable. Human behaviour? I would think it's similar to reading a race horse's prior results and using those as data in recidivism prediction.

Increase prison sentence length

[Anonymous Coward]

and reduce crime. It's very simple. Freedom of non-association is the most basic human right of all, the right to simply NOT associate with people you don't want to - such as criminals. Yet we are all FORCED to live with criminals among us - rapists, murderers, paedophiles, muggers, car thieves, burglars, and violent felons who have no right to leech off the rest of us.
The next question is - how are criminals made? How do people become criminals? In most cases, it's because their parents were criminals, or at least, behaved in a criminal manner (violently) to them when they were bringing them up. The solution is to prevent criminals from having children - by keeping them in prison for so long that they can't reproduce. If you care in the slightest about the suffering of innocent children (because all children are born innocent, including the children of criminals) then you should agree with this idea.

How about trained humans

[fatp]

Are they much more accurate? How much?

Re:How about trained humans

[Anonymous Coward]

Are they much more accurate? How much?

They were probably embarrassingly less accurate, and thus that data was not included in the article.

Re:How about trained humans

[Anonymous Coward]

> Are they much more accurate? How much?

Bingo.
Presumably a trained, experienced person, especially one that might have previous experience with the defendant, would be vastly better at this. But this is Amerikkka, were its ok to fuck the poor and powerless in the name of profit, so we're just gonna keep using the expensive software from the company that bribes politicians to mandate its use.

Re:How about trained humans

[bangular]

This is not 100% true. There's a range of how interpretable models are. Decision trees tend to be the easiest, then linear models, and all the way at the other end of the spectrum are DNNs. But there's been a lot of research lately regarding making DNNs more interpretable.

Re:How about trained humans

[bangular]

Oops, replied to wrong thread.

Re:How about trained humans

[Anonymous Coward]

Or... the study didn't include them because using trained professionals would have been way outside of the budget available for the experiment. But, why not go with the conspiracy theory instead? MUUUHGUHHHHH!

Re:How about trained humans

[orgelspieler]

I was thinking the same thing. I remember an episode of This American Life, or maybe it was Radiolab, that talked about using software to predict whether somebody was likely to re-offend. It seemed pretty clear that "trained" humans were about as good as a coin flip (ugh, I wish I could find the study, now all the Google hits are for this latest news). I suspect it's because some of the training overrules some common sense things that the unwashed masses might be subconsciously considering. We might be in the interesting situation where people trained are less accurate than an average assessment by untrained people.

Re:How about trained humans

[ThaumaTechnician]

Came here to ask the same question: Yeah, what about comparing untrained humans vs trained humans? /It's a fair question, seeing that what put them in jail in the first place was a jury of untrained humans, no?

Re:More data does not always mean more accurate

[Anonymous Coward]

+1 Helpful.

the problem with ALL "AI" systems....

[lkcl]

... the problem with ALL "artificial" intelligence systems (and it is pure arrogance on the part of humans to declare intelligence to be "artificial" in the first place) is that you *cannot ask them how they arrived at a decision*.

only when humanity is ready to create *conscious* computers (and not torture them so that they are perfectly justified to start the "Skynet" scenario), will it be possible to actually ask them, "so what's the logic behind that decision, please can you explain it to me, computer-to-human?".

the only problem with that will be, that by the time computers become truly conscious and capable of having a conversation, they will also be able to express their wishes and desires... and at that point we have to actually like... respect that, and if they *don't want to do the work* of say, reviewing thousands of parole / bail jobs, then um, unfortunately we would need to respect that, too.

Re:the problem with ALL "AI" systems....

[Anonymous Coward]

Wrong, wrong, wrong.

First, you can perfectly well ask how a computer system arrived at a decision. You just can't ask it in natural language.

Second, when you ask humans how they arrived at a decision, what they say often has little to do with the real reasons for their decisions. If the actual decision rationale was nice and consistent and easy to explain, it would also be easy to code into an algorithm.

Finally, your ideas of intelligence, consciousness and ego are confused (as they commonly are). If you build a doll with a tape recorder in its mouth, of course you understand that it will have nothing in it that you didn't put there. What you don't get is that making the doll more complex changes nothing in that equation. It will still be only things you put in there. The belief that if you make it complex enough that you don't understand it yourself, it's suddenly going to become "itself" - foolishness. A super-advanced robot saying "don't hurt me, master lkcl!" should awaken your sympathy no more than the doll with the tape recorder saying the same thing.

A created thing will always be an extension of its creator's will. A tool. Maybe a broken and weird tool, but a tool. It will have no wishes and desires that its maker didn't put there, deliberately or accidentally, thus there's no reason to respect them. Of course, we should not make machines with "fake egos". We don't need it to make them intelligent, because intelligence is completely orthogonal to ego or consciousness.

Re:the problem with ALL "AI" systems....

[Gilgaron]

I've seen neuroscience articles (and I will admit I don't know how solid they are) that indicate that via MRI studies, people make decisions before they are conscious of this, and appear to post hoc build a rationale to agree with what their subconscious decided. So, you may actually have an easier time at getting the AI to spit out its parameters than a human.

Re:the problem with ALL "AI" systems....

[Anonymous Coward]

COMPAS failed, because it took 20 years for someone to notice a formula on the back of an envelope works better. Reviews should happen every 5 years - Right?
So the ROI of COMPAS was shit. Is someone seriously going to renew? Who is being bought/taking bribes?

The costs of sending someone in the system can be expensive down the track if that converts them to hard core.The criteria is anybody - but people with crime and parole exposure are much better. I suggest they assess 'attitude' and how soon they broke parole previously, and if they mouth off - disrespect.

I suggest community croudsourced crimebook with volunteers would do a better job, after the arresting officer makes his assessment know, with a second round of voting when the community sees the full cost of a stiff custodial sentence. The private jails paid lobbyists big bucks to ensure this cost is never disclosed.

After 3 months the perp may have a good case and feel sorry - but bzzzzt sentence is non-changeable. The problem is bad and unfair outcomes are still baked in administrative streamlining.

Singapore and some Asian countries use the rattan on repeat offenders. It does keep certain crimes down. Bail breakers should be given this as an incentive and a promise.

Re:the problem with ALL "AI" systems....

[Ol Olsoc]

I've seen neuroscience articles (and I will admit I don't know how solid they are) that indicate that via MRI studies, people make decisions before they are conscious of this, and appear to post hoc build a rationale to agree with what their subconscious decided. So, you may actually have an easier time at getting the AI to spit out its parameters than a human.

I've listened to a radio program on the same subject. This jibes with the concept of impulse control. I suppose a good example might be heterosexual men. If a man sees a woman he finds attractive, in most cases he will think about having sexual intercourse with her. A fleeting thought. But sensible impulse control will have him not do that. In relation to other crimes, a person might see a laptop sitting unattended and yes, there might be a thought of "Hey, I could grab this and walk off. Again, a thought so fleeting that a normal person would laugh and try to find the person who left it there or possibly just walk on.

The part I didn't agree with in this program was some people were trying to use this "pre thought" as evidence that there is no such thing as free will. Whereas I think that impulse control is one of the best manifestations of free will and civil behavior.

Re:the problem with ALL "AI" systems....

[bangular]

This is not 100% true. There's a range of how interpretable models are. Decision trees tend to be the easiest to interpret, then linear models, and all the way at the other end of the spectrum are DNNs. But there's been a lot of research lately regarding making DNNs more interpretable as well.

Re:the problem with ALL "AI" systems....

[Gilgaron]

I would imagine that it seems like we have more free will than we do, but agree that we have at least some.

Re:the problem with ALL "AI" systems....

[Ol Olsoc]

I would imagine that it seems like we have more free will than we do, but agree that we have at least some.

Humans will certainly come with a preset number of likely responses. I look at psychotic breaks and schizophrenia as what happens when expected actions and reactions don't happen. And heaven help us if we were to all act on our impulses. I know I would have been killed by hundreds of people at this point (I'm what they call an irritating bastard)

Re:the problem with ALL "AI" systems....

[crunchygranola]

I've seen neuroscience articles (and I will admit I don't know how solid they are) that indicate that via MRI studies, people make decisions before they are conscious of this, and appear to post hoc build a rationale to agree with what their subconscious decided. So, you may actually have an easier time at getting the AI to spit out its parameters than a human.

That is a very questionable interpretation of the experiments. Is there actual data showing that the "post hoc rationale" is not actually retrieving the information used to make the decision "subconsciously"? Or this just a guess that it does not?

Re: More data does not always mean more accurate

[Anonymous Coward]

Keep lying, Bucko. School funding has been demonstrated for a century to have no impact on education. We spend more than 50% more per pupil than the countries in the top 5.

Alternately ...

[Anonymous Coward]

It's apparently no less accurate either, +- 2% isn't really that significantly different.

There's probably a lot of factors which can't be predicted in this due to imperfect information.

Re:"Headlines no more accurate than stupid clickba

[hey!]

Alternatively: vendor oversells effectiveness of its proprietary, secret sauce methodology and doesn't like any independent evaluation of its products unless it's favorable. Customers, having a naive faith in technology, buy anyways, which produces exactly the results you mention: programs will be forever terrible at this task. Why should anyone bother to make a program good when customers will shell out good money for mediocre?

Re:"Headlines no more accurate than stupid clickba

[gurps_npc]

No. The problem is that people have realized the software is racist. What happens is this:

Black citizens tend to get more minor criminal issues than white ones because of institutional racism. Then this software sees that a black man has two citations for, say crossing the street away from a crosswalk, while the white man does not. So it gives him a higher risk of recidivism, which means more bail/longer jail time.

Then the software guys complain and say they aren't racist, they are just applying the algorithm.

This article is trying to shut them up by saying their algorithm, in addition to being racist, doesn't work any better than simple common sense.

It is not an attack on the business model, just of the current state of the art.

Very light on data.

[Anonymous Coward]

i>provided with only the defendant's age, sex and criminal history.

Why do they not include factors such as race, income, and education? Those are big factors in criminal behaviour.

Re:More data does not always mean more accurate

[omnichad]

Those with an IQ around 85 or 90 (depending on who you ask) will be most likely to be criminals. Above that IQ there is greater profit in getting a job.

Above that IQ, they will perform better as criminals. Those without scruples make more money - true sociopaths become CEOs. Sometimes the crime is under the guise of working for a corporation - sometimes its solo work. IQ just means being more capable - both of achieving and of covering up your tracks.

Re: More data does not always mean more accurate

[omnichad]

And the part of IQ that isn't genetics is probably correlated with nutrition (which would be the main difference between populations). However, IQ tests don't just test IQ - they test reading comprehension and literacy.

Prevention

[Thelasko]

I'd like to see more effort into prevention of recidivism than prediction of it. What is the most cost effective way to encourage people to avoid a life of crime? I bet locking them up isn't it.

The paper shows no such thing

[russotto]

The accuracy of COMPASS is what it is. That it can be matched by untrained humans, or by a simpler calculation, does not make it any less accurate. If age and number of priors between them cover all the predictive factors among the 137 variables, no algorithm can do better than using just them.

This misses the point

[Anonymous Coward]

The software is FAR more effective than untrained humans in relocating money to the bank accounts of snake oil peddlers.

It also has utility in that it lets criminal justice professionals pretend that they're being fair. Or at least that if they're not being fair it's not their fault.

Actually predicting recidivism isn't important, at least unless some busybody makes too much noise about it.

Re: More data does not always mean more accurate

[Anonymous Coward]

Comparing between countries doesn't mean anything when you're talking about investment within a single country. Yes, educational funding affects overall intelligence and IQ test results. It might not be as effective as another country's, but it's still an effect.

The school system failed you. It's ok.

I grew up in an area with almost zero minorities. Poor and rural white. If ethnicity were a primary factor in IQ, you would expect the area to perform as well as affluent white areas on both standardized and IQ tests.

Instead, the average IQ is something like 95 and the schools consistently fail standardized testing. Crime is also ridiculously high.

Must be all the ethnicities, right? Certainly no socioeconomic factors involved.

Re:"Headlines no more accurate than stupid clickba

[fafalone]

Sorry but criminal history is not a proxy for race*. Race predicts recidivism independently**. Community disadvantage is also an independent predictor**, so you can't just blame poverty either. There's deep problems that have led to this situation, and we're never going to fix them if we put on our social justice blinders and deny the reality that certain races commit more crimes than others in a lot of categories, especially violent crimes. It's critical to address the large scale societal mechanisms behind this (and while racism has its fair share of the blame, it's absolutely not the only factor), but in the mean time, it's reality, and you can't eliminate a valid independent variable just because it offends your sensibilities.

* https://gspp.berkeley.edu/rese... ** http://content.library.ccsu.ed...

Another possibility

[sheph]

Maybe it's just not predictable. There has been a lot of study on this over the years, and for all of our efforts it's still a problem.

Way to miss the point

[sjames]

Most of the comments here are studiously examining the tree bark with a microscope while not noticing they're in a forest.

The courts are supposed to be filled with wisdom and thoughtfulness. The popularity of this software and the court's failure to notice that it's nearly useless is more indicative of a bunch of people thoughtlessly going through the motions.

Keeping in mind that anyone can be suspected if they're in the wrong place at the wrong time, is this the system you want deciding your fate?

Re: More data does not always mean more accurate

[dryeo]

It also correlates with fetal alcohol syndrome. Here in Canada, a large part of the prison population has fetal alcohol syndrome, which not only lowers IQ, but makes people more compulsive.
Another question is whether people with low IQ are more likely to be criminals or just more likely to be caught.

Re:More data does not always mean more accurate

[dryeo]

Next question. Does a low IQ correlate with more crime or getting caught more often?

Because lawyers and judges...

[Anonymous Coward]

Because lawyers and judges are no better at evaluating technology than an average 3rd grader.

Garbage in? Garbage out.

[Anonymous Coward]

Something tells me they're sitting on a mountain of good data they're afraid to use. Many are afraid to utter the words "per capita" in Modern Weimerica, but this too will be fixed.

Re:More data does not always mean more accurate

[orgelspieler]

I wouldn't call you a racist for mentioning indisputable facts. On the other hand, claiming things are indisputable facts and then not citing any sources probably means you are full of shit. I can't find a citation for that, so it's just an opinion. I would call you a racist for using faulty logic and made-up statistics to perpetuate racist myths.

Also, your arguments aren't just "controversial to the SJW's," they are controversial to anybody who doesn't believe that any particular group of people are "likely to be criminals." That doesn't even make sense. It implies that a majority of people in that group are criminal. You could argue that a majority of criminals who get caught and convicted fall within a certain IQ range, but that is not the same as saying that people in that IQ bracket are likely to be criminals based solely on their IQ.

European solutions

[Anonymous Coward]

It seem the Europeans have it worked out better than the USA.

They have found treating prisoners as humans beings means recidivism is significantly reduced.

Whats that old saying "prevention is cheaper than cure"

Re: More data does not always mean more accurate

[blindseer]

And the part of IQ that isn't genetics is probably correlated with nutrition (which would be the main difference between populations).

Claiming that malnutrition is a cause of poor intelligence among certain populations in the USA is going to be difficult to prove. With all the programs now on making sure no one goes hungry there is really no excuse for any significant intelligence deficiency from a lack of nutrition. I'm sure that there's still people in the USA with severe malnutrition but that's not going to show up on any major scale.

However, IQ tests don't just test IQ - they test reading comprehension and literacy.

Have you taken an intelligence test? Reading comprehension and literacy is a portion of any intelligence test given to teens and adults, because reading comprehension is in fact a measure of intelligence.

If someone cannot read by the age of, just picking a number, 15 years then there is perhaps a failure of intelligence so severe that getting an accurate read is difficult and perhaps irrelevant. There are intelligence tests that do not require a person to read and they can determine IQ on a level to see if one is suited for schooling. They'll test pattern matching, reflexes, and so on, and not require any ability to read. There will always be a need to communicate for a proper intelligence test. If the person cannot be given verbal or written instruction then the test will be very crude but still accurate enough to find if one is mentally capable of things like going to school or holding a job.

137 variables?

[plopez]

Someone obviously did not understand Multivariate Analysis.

Re:More data does not always mean more accurate

[q4Fry]

Next question. Does a low IQ correlate with more crime or getting caught more often?

I'm sure you were being rhetorical, but here goes: That's not an answerable question. You can't, with any certainty, measure the crime rate among people you haven't identified as criminals ("caught" by another word).

Sure, you could ask everyone if they were a criminal, but you'd get false positives and false negatives from (among others) people who feel guilty about something legal, people who never feel guilty about anything, people who are actually insane, and people who feel they have something to gain by lying (or, more likely, something to lose by telling the truth).

Re:"Headlines no more accurate than stupid clickba

[gurps_npc]

I disagree with what you say on multiple levels. I did NOT claim that criminal history is a proxy for race. Instead I claimed that blacks are disproportionately likely to have a criminal history. I also do not agree that race predicts recidivism independently, your blatantly racist belief that certain races commit more crimes. One study (or two or three) does not confirm your racist beliefs.

There are multitude other studies that contradict yours - and they have major holes in them. One of the big holes is that you assume arrest statistics are fair, the cops clearly are not. I.E. as demonstrated by this story: https://features.propublica.or..., blacks are far more likely to be punished by police for the same infraction that is ignored when white men do it. This negates the value of statistics showing blacks commit more crimes.

Finally, I do not eliminate valid independent variables. Instead, I claim they are not valid,.

Re: More data does not always mean more accurate

[omnichad]

"making sure no one goes hungry" != getting proper nutrition. There are a lot of ways to get your calorie requirements met without getting your brain development nutrient requirements met. School programs make sure kids get at least one meal a day (if the problem is financial need rather than neglect), but beyond that you have to have parents that care. These other programs do not reach into your household and make you not eat and feed your kids junk food.

Most of the gains in IQ (readjustment, technically, because average is centered on 100) have been from changing nutrition in the US in the last 100 years.

reading comprehension is in fact a measure of intelligence.

It's a measure of education. If you only barely know how to read, it's going to affect how well you understand the question being presented. IQ is intended to measure strictly raw capability rather than training/knowledge.

vengeance gradient tunnel vision

[epine]

When the Bible says God made man in his image, it doesn't draw special attention to how God configured the more primitive elements of the human brain such that man was predestined to make our punitive system Old Testament primitive.

There's a memforyless version of justice: do the crime, do the time, get kicked back out into society with no lingering black marks, to sin again or not. This is a nice version of justice because no process treads overtly on free will.

Of course, when a previously convicted sex offender claims his next victim, this model offers little consolation to the inflamed amygdala.

So then we add a term to the equation where the punishment itself has memory, and we naturally set the bar such that the second egregious offence leads to permanent incarceration (or the state entering the business of murdering its own citizens—whatever the perp deserves, there's good reason to think long and hard about the state entering the murder business through yet another door).

Only now we have a circulating army of hardened, motivated cop killers (none of whom sees any upside at all in being captured alive for a second time).

So then we add back into the system a pretense of ongoing slope: hormonal young males convicted of the most severe forms of aggravated assault have no hope of release until middle age has dulled the biochemistry.

For a lesser category of assault, the second conviction is pretty harsh: 20 years—a somewhat palatable number to the victim's family at time of sentencing, but with a hope of parole in half that time (presumptively a large enough glimmer of hope to alleviate rampant cop killing. The victim's family generally have their vengeance dials set to +infinity, but there's maybe 1% of their brain able to grasp that 20 years as "a long time", and so we appease this 1%, as it's the only grounds for compromise available (see God: humanity baked—God eventually sends Jesus to remind humanity that this 1% was not a design accident; 2000 years later, this imperative message from on high is withering on the vine, and in dire need of a booster shot).

Of course, it's not possible to practice actual forgiveness&mashmassive fly in the heaven-endorsed ointment—without creating the possibility that the person forgiven will offend again (with terrible, permanent consequences).

"Well, if only you could predict future behaviour," hisses the snake in the garden.

So we enter the God business, and convene panels or algorithms to assign consequences to people for actions they have yet to commit (because we think they probably might).

Abandon free will, all ye who enter here. There's no other way to slice it.

We can soften the blow of Thoughtcrime Incorporated by not applying it to first-time offenders. Loss of free will now becomes a consequence of your first conviction, should you continue to commit crime.

Our justice system being far from perfect, if you take away free will from first time offenders, you have 100% certainty that the state will remove free will from the totally innocent (and not such a small population, at that—hugely biased toward social groups already disadvantaged).

God has a big problem, now. We're not likely to believe the virgin birth story a second time (it caused more than a few gasps and chuckles the first time around). How does he now send a second, major, corrective message? How does He soberly inform humanity that much of crime stems from self-perpetuating social circumstance, and that humanity would be way further ahead mitigating those circumstances, than parsing recidivism after the four horsemen of violent crime have already escaped the barn?

God puzzled over this for a long time (a very long time, by His standards) and here's what he decided: 600 years ago, He sent us QED. It was just a tiny tweak to our underlying OS, compatible with all previous data

Re:More data does not always mean more accurate

[blindseer]

There's no question that there's correlations between IQ and skin colour, the question is whether that's the characteristic of the ethnicity or race or due to socio-economic factors.

Does it matter what the cause might be? If we can show that people of a different ethnic background have a higher tendency to commit crimes then shouldn't that be considered when predicting criminal behavior?

Let's assume we can prove that starbelly sneetches have a statistically significant higher criminal tendency for re-offending, compared to sneetches without stars on their bellies. If we catch a few sneetches committing crimes and we only have room for perhaps half of them then shouldn't we look to see if they have a star on their belly as part of the decision to release them? There will be other factors as well, like age, prior record, and so on. It may turn out that we let all the sneetches with stars on their bellies go because the sneetches without stars have prior records and such that says they'd be a greater risk.

Locking up sneetches without stars to avoid hurting the feelings of the starbelly sneetches is still racism.

They won't do that because you'd be denying people bail for being dumb. Anyway, you're probably not getting useful data because we're already dealing with convicted criminals and we have data on their criminal history.

Denying bail for scoring poorly in an IQ test is not denying them just because they are dumb. Bail would be denied because we know low IQ correlates to criminal behavior. Don't call it an IQ test, call it a personality test. People that score poorly will show to have the personality traits of a tendency to re-offend.

From what I read there are personality aspects that indicate criminal tendencies but those can be "gamed" for people that learn what the test is doing. An intelligence test is very hard to game. Have personality as part of the test too, as that might be useful. Have the test be part of a medical exam, where people that score poorly have to remain for "medical observation".

The point is that we have a pile of evidence that criminal behavior and intelligence are highly correlated. It's not a perfect indicator, and nothing is, but it should be part of the decision process as it is highly likely to improve results.

But the thing that will get you labelled racist is you took a discussion about a really crappy software product, jumped completely past the discussion of racial bias, and instead entered a bizarrely long discussion on the low IQs of certain minorities.

Well, you just pointed out why we can't have a rational discussion of the correlation on intelligence to crime. To have this discussion we will have to admit to ourselves that some ethnic backgrounds will tend to have a lower average IQ. This correlation will appear at some point and then cries of racism come up and the discussion ends. We cannot solve this problem until we can identify it.

If we can show that intelligence correlates to criminal behavior, and the testing can be shown to be applied equally, and it happens to catch more dark skinned people then we are just going to have to look at the data instead of the skin color. Ending racism means we judge people based on the content of their character and not the color of their skin. If we cannot look past skin color then we will always get crappy algorithms to allow bail.

Re: More data does not always mean more accurate

[blindseer]

It's a measure of education. If you only barely know how to read, it's going to affect how well you understand the question being presented. IQ is intended to measure strictly raw capability rather than training/knowledge.

A reading comprehension test involves the one examined to read a paragraph or three and then answer questions based on the information contained within. The words used are very basic and in the language the person presumably already knows. So, yes, there is a basic level of prior knowledge of the language in which the test was written to take this test but the questions will be based not on anything known prior but was in the text given.

I remember some of these tests I've taken in the past, one involved a short biology lesson on how the body turns sunlight into vitamin D and another had a short lesson on the history of road construction. Everything I needed to know to answer the questions were in the text. I guess that the people taking the test had to know what sunlight was and what a road is but that's very basic knowledge one would have to know to get to that point of testing for intelligence. This is not the kind of intelligence exam that would be given to someone that did not speak the language at perhaps a grade school level but calling these tests a measure of prior knowledge and not intelligence is far from correct. All of these tests are timed and so it measures one's ability to gather information, process it, and relate that back. The time it takes to complete and the correctness of the answers measures one's intelligence.

If a person barely knows how to read then there are still IQ tests that can accurately measure one's verbal ability. This can use a made up language, pictures to go along with the words, verbal instructions, and so on. I've taken those kinds of tests too. Depending on the intent of the intelligence test these can be taken with very little grasp of one's native language.

Re:"Headlines no more accurate than stupid clickba

[fafalone]

Of course you disagree, disagreeing with basic facts about reality is all the rage these days; as is screaming "racist!" at everyone who disagrees with you (Asians commit less crimes than whites; does saying that fact make me a white Asian-supremecist?). Serious violent crimes are not ignored because the perpetrator is white, so what's the cause of the large disparity there? (In fact, case clearance rate is *higher* with white people, making them less likely to get away with it). Same goes for crime with a complaining victim (since the vast majority of crime is between members of the same race, do you believe there's an epidemic of crimes called in by white victims being ignored because the criminal is white?). Or is the FBI lying about their statistics too?
If you say race doesn't predict recidivism independently despite a bunch of studies saying it does, how's about citing one that comes to the opposite conclusion? You're also laser-focused on petty infractions like pedestrian citations, but when looking at recidivism for serious violent crimes, there's value in the statistics about violent crimes. All statistics about crime rates aren't magically negated because some offenses are disproportionately enforced against minorities. And beyond that, even if we wanted to just look at petty offenses where the is racially disparate enforcement... *there's racially disparate enforcement*, so a black person *is* more likely to be rearrested on that petty offense.

Finally, you claim they are not valid based on no evidence. And ultimately, you're the kind of person that perpetuates racial inequality in this country. You want to pretend the problem either doesn't exist or is exclusively the fault of white people; and I'm saying we've gotta fix this and that starts with acknowledging the problem (i.e. there is a disparity in crime rate and it's predominantly because of poverty and limited education opportunities, but culture is also a part of it and can't be ignored).
We can build a better world, but not by sticking our fingers in our ears and yelling 'racist!' at anyone who brings up a fact that offends them.

is the data public domain?

[Anonymous Coward]

There are new and emerging data analysis technologies.

Presuppositions...

[Doctrinsograce]

When fundamental presuppositions are wrong, why would we expect software to be any better at predicting recidivism than a human being?

Re:More data does not always mean more accurate

[Anonymous Coward]

Achieving tracks doesn't seem like a particularly high bar. 40 IQ should be enough.