Slashdot Mirror


Wisconsin's Prison-Sentencing Algorithm Challenged in Court (engadget.com)

"Do you want a computer to help decide a convict's fate?" asks Engadget, telling the story of a Wisconsin convict who "claims that the justice system relied too heavily on its COMPAS algorithm to determine the likelihood of repeat offenses and sentenced him to six years in prison." Sentencing algorithms have apparently been in use for 10 years. His attorneys claim that the code is "full of holes," including secret criteria and generic decisions that aren't as individually tailored as they have to be. For instance, they'll skew predictions based on your gender or age -- how does that reflect the actual offender...?

[T]he court challenge could force Wisconsin and other states to think about the weight they give to algorithms. While they do hold the promise of both preventing repeat offenses and avoiding excessive sentences for low-threat criminals, the American Civil Liberties Union is worried that they can amplify biases or make mistakes based on imperfect law enforcement data.

The biggest issue seems to be a lack of transparency, which makes it impossible to determine whether convicts actually are receiving fair sentences.

39 of 228 comments (clear)

  1. Yes please by penguinoid · · Score: 5, Insightful

    If you think a computer is biased and unfair and makes decisions using secret criteria, wait till you meet a human!

    --
    Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
    1. Re:Yes please by Anonymous Coward · · Score: 2, Funny

      Can we close the comments section now please he said everything we need

    2. Re:Yes please by Archfeld · · Score: 5, Insightful

      There is no AI involved, just some code written by humans. A computer cannot express compassion. IMHO Compassion is the highest virtue, and the first to be discarded. Hope can be rekindled, faith restored, love reunited but compassion once lost is rarely seen again. Even the guilty deserve compassion, it is that which elevates man from beast. We do that which is necessary but it should NOT stop us from being sorry it has to come to that.

      --
      errr....umm...*whooosh* *whoosh* Is this thing on ?
    3. Re:Yes please by Actually,+I+do+RTFA · · Score: 3, Insightful

      People are complaining because the computer is biased, and unfair, and uses secret criteria and it disagrees with human intuition.

      But algorithmic bias is worse than human bias, because people know other humans are biased, but give a pass to computers.

      --
      Your ad here. Ask me how!
    4. Re:Yes please by JoeMerchant · · Score: 5, Insightful

      The advantage of a computer algorithm is that it can be analyzed, refined, improved, and relied upon to function more or less as well as it did in the past (assuming the new inputs aren't radically different from the previous ones.)

      Now, judges - they come to their position through politics and lawyering, two selection criteria that would seem to get you the last people you would want to determine what is fair or reasonable for "the people at large." Even 200+ years ago this was recognized and is the basis for "trial by jury of peers" laws, and everyone knows that amounts to a random lottery style decision, but it's still better than leaving it up to a judge.

    5. Re:Yes please by penguinoid · · Score: 4, Funny

      Having the computer show compassion is easy.
      if (criminal.gender == female & criminal.attractive == true)
            showCompassion = true;

      --
      Don't waste your vote! Vote for whoever you want, unless you live in a swing state it won't matter anyways
    6. Re:Yes please by lucm · · Score: 4, Funny

      Having the computer show compassion is easy.
      if (criminal.gender == female & criminal.attractive == true)

            showCompassion = true;

      that looks like the algorithm they use in custody battles.

      --
      lucm, indeed.
    7. Re:Yes please by taustin · · Score: 3, Insightful

      The advantage of a computer algorithm is that it can be analyzed, refined, improved, and relied upon to function more or less as well as it did in the past (assuming the new inputs aren't radically different from the previous ones.)

      Not when the entire process is kept secret. Which is the complaint here.

    8. Re:Yes please by mwvdlee · · Score: 2

      Atleast the "secret criteria" could be easily fixed by making it output it's reasoning.

      --
      Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
    9. Re:Yes please by invictusvoyd · · Score: 2

      showCompassion( )
      {

      haveErection ;
      sleep 1 ;

      }

    10. Re:Yes please by Anonymous Coward · · Score: 4, Insightful

      Most of the algo's used are "privately developed", and thus not released because that would involve revealing "business critical information" (which obviously trumps the interests of the person being sentenced on the basis of it). Because the primary aim is to provide private entities with contracts/work/govt money.

    11. Re:Yes please by cdrudge · · Score: 4, Funny

      My compiler says that criminal.attractive is undefined. Can you post the code for that property so that the system knows what attractive is.

      Also, I think criminal.gender is deprecated. These days, I think you're suppose to use criminal.birthGender or criminal.identifiedGender.

    12. Re:Yes please by JoeMerchant · · Score: 3, Insightful

      If the algorithm is secret, then it's worse than trusting a judge who can be removed from the bench any number of ways.

      Judges are supposed to follow the law, which is public knowledge. If we're algorithmically systematizing sentencing, that algorithm needs to be public too - before the evidence comes out in the algorithm's output.

    13. Re:Yes please by mrchaotica · · Score: 3, Informative

      You missed the "... || criminal.wealthAndPower == high" clause.

      Also, you did a bitwise and instead of a logical one.

      --

      "[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz

    14. Re:Yes please by GuB-42 · · Score: 2

      The "secret criteria" is probably along the lines of "criteria X statistical significance is 0.02456 and a weight of -0.12466 improves the predictions by 0.03154". The criteria is probably not secret but rather completely opaque for the human mind.

    15. Re:Yes please by Hognoxious · · Score: 2

      if prisonOperator.hasPaidUs == true {
                  prisoner.sentence = sentenceMax }

      --
      Confucius say, "Find worm in apple - bad. Find half a worm - worse."
  2. Justice is blind and buggy by justcauseisjustthat · · Score: 2

    In the US there are already way too many people in jail, letting buggy software sentence people is a joke.

    1. Re:Justice is blind and buggy by JoeMerchant · · Score: 4, Interesting

      If the software is open source, transparent, and clear in how it functions, I'd rather have sentencing software than a judge in most cases.

      The problem comes when you get politicians meddling in the software, writing in "zero tolerance" code and "mandatory minimum sentences" - they've done this already in the legislatures, tying judges hands in sentencing decisions. With the data available: prior convictions, credit scores, family ties, etc., software can determine probable outcomes of lenient sentences.

      Yes, I'm saying credit score is a point for consideration in sentencing - not decision of guilt or innocence, but once guilt has been established, credit score tells about a person's history of making good on commitments, and should be a strong predictor of their likelihood of meeting terms of a suspended sentence, probation or parole.

    2. Re:Justice is blind and buggy by jopsen · · Score: 2

      Combating repeat offenders with longer prison sentences, ie. essentially locking them up... is not a solution.

      This is so misguided on so many levels, I don't even know where to start.

    3. Re:Justice is blind and buggy by mrchaotica · · Score: 2

      No, what we should really be doing in the US is holding you fuckers at the CRAs accountable for your systemic pattern of libel. CRAs should be obligated to vet any information submitted to them for accuracy before passing it along to others or using it to calculate a credit score.

      --

      "[Regarding the 'cloud,'] ownership was what made America different than Russia." -- Woz

    4. Re: Justice is blind and buggy by JoeMerchant · · Score: 2

      That's called a credit dispute - I've had them, I'd be suspicious of anyone over the age of 35 who hasn't had at least one by that time.

      If you've had one run-in with one crappy landlord, that should exonerate any impact that single case had on your credit rating (I managed to get through my crappy landlord experience without a court case, but many good tenants before me did not, and the algorithm could actually look into that and forgive them their head-butting since the pattern is clear that the other party is initiating trouble.) If you have a string of six evictions, with heavy visible consumer spending but you're making clear choices to just not pay your landlords - yeah, that's not so good as a recommendation that you'll follow through with a commitment to not repeat your other criminal activity. In-between, that's where it would be nice to have an algorithm taking into account things like prison crowding, current economic conditions and liklihood you can find gainful employment, etc. instead of a judge making a call based on "his gut feelings" which can often include race, or other personal prejudices of the judge.

    5. Re: Justice is blind and buggy by akical0118 · · Score: 2

      My parents are millionaires, I've never needed a loan, I pay cash for everything I'm listed as self employed, I own a bar and have 20 employees, yet my credit score is 520 because my credit record is empty, the bar was paid for cash

    6. Re:Justice is blind and buggy by JoeMerchant · · Score: 2

      My first black stain came from student loans that got erroneously put into repayment after my freshman year, showed multiple 120+ days past due on the report, nobody informed me until after graduate school almost 6 years later. These things get explained and the final decision takes those explanations into account.

      So, if the algorithm says: "650, maximum sentence." that's a bad algorithm, use a judge instead. If the algorithm can take into account circumstances, disputes, resolutions, etc. then it might be less prejudiced than a human judge.

  3. Gender? by Anonymous Coward · · Score: 2, Interesting

    Wait, it's iCurrentYear, why would we be giving any weight to gender in sentencing? Men and women have equal rights, yes? So why should say, someone who raped a teenager over 50 times get off scott free just because of their gender?

  4. Algorithms for sentencing ... by CaptainDork · · Score: 2

    ... and I thought Facebook was fucked up.

    --
    It little behooves the best of us to comment on the rest of us.
  5. Re:And this is different how? by PopeRatzo · · Score: 2, Insightful

    don't see Brock Turner complaining much.

    Actually, Brock Turner's dad absolutely did complain that 6 months was too severe of a sentence for his special snowflake rapist son.

    --
    You are welcome on my lawn.
  6. The real issue is lack of transparency by cowtamer · · Score: 5, Insightful

    While the idea of using an algorithm to sentence a human being is bone-chilling, you might be able to justify this as a "formula" for sentencing -- which, of course, merits its own debate.

    What is unconscionable about this is the fact that it's a SECRET algorithm. As in closed source. Essentially a secret law.

    This has no place in democracy.

    (Also, any algorithm which ingests statistical and demographic data is bound to come up with unpalatable and/or spurious demographic correlations (since there is a causal link between poverty and crime and a historic link between race and poverty) which I wold rather have society refrain from codifying -- in law or in actual computer code).

    1. Re:The real issue is lack of transparency by K.+S.+Kyosuke · · Score: 2

      (Also, any algorithm which ingests statistical and demographic data is bound to come up with unpalatable and/or spurious demographic correlations [engadget.com] (since there is a causal link between poverty and crime and a historic link between race and poverty) which I wold rather have society refrain from codifying -- in law or in actual computer code).

      Heh. That actually reminded me of Minskys article on "Why programming is a good medium for expressing poorly understood and sloppily formulated ideas". Why, it even speaks of "A program as a court of law"! But I would like to see some experimental computerized justice - not in the sense of this thing that is being criticized, rather as a research vehicle - to simply see if it uncovers some of the "poorly understood" and "sloppily formulated" parts of our justice systems. Perhaps it could enable us to right some wrongs in the future if we find out the things that we don't know that we don't know.

      --
      Ezekiel 23:20
  7. Statistics by Hognoxious · · Score: 4, Insightful

    For instance, they'll skew predictions based on your gender or age -- how does that reflect the actual offender...?

    It doesn't and it's not intended to.

    It's statistics and probabilities, just like how insurance premiums work.

    --
    Confucius say, "Find worm in apple - bad. Find half a worm - worse."
    1. Re:Statistics by NotInHere · · Score: 4, Insightful

      like how insurance premiums work.

      That *is* skewing predictions based on gender and age.

      If a computer program deduces from the fact that you are male that you will live a shorter time, and this makes women receiving lower payments due to them being more likely to live longer.

      As this discriminated against women, this practice was swiftly forbidden by the EU. It probably wouldn't have been forbidden if it discriminated against men, but that's another issue. It was actual gender based discrimination.

      If you let a computer program take factors like skin color or religion into account, that computer program will arrive at similar conclusions that humans arrived:

      * blacks do more crimes per thousand people than white people
      * highly educated people do less crimes than less educated people
      * More muslims commit terrorist attacks than non-muslims

      This doesn't mean that black people are ape-like, or that less educated people are automatically criminals. It usually just is a correlation, which doesn't imply causation. If you took a black person, and remove them from all the racist and discriminatory influences, AND the lower social status (that has been caused by lots of discrimination before), then won't end up with a higher crime likeliness than if you took a "normal" white person. Its probably even more caused by the social status than discrimination.

      As for muslims, its simply caused because there is no powerful christian group that radicalizes christians to commit terror attacks. If you take the radicals out, the average muslim is as peace-loving as the average christian or jew.

      Denying the existence of these correlations to protect from discrimination is wrong. This is just fuel for people who claim that the existence is denied because people want to "cover up" something, or because people are too dumb, usual criticism by people like trump. It is also wrong though to do actual discrimination, like israel does it, or trump proposes.

    2. Re:Statistics by Ichijo · · Score: 2

      If convicts could get out of prison as soon as an insurance company will insure them against repeat offenses, then all sentences would be the same: they would be the grown-up version of "sit in the corner until you've learned your lesson."

      Can you think of a better incentive to rehabilitate than to know that you'll get out as soon as you're cured? Is there any good reason to keep people locked up longer than that? Is it wise to release people before they've rehabilitated?

      --
      Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
  8. Re:The 'real' software by Firethorn · · Score: 4, Insightful

    Why don't they use the fundamentals used in developing the algorithm to identify and eliminate the root causes of crime (through rehabilitation/counseling) and thus reduce crime in highly prone areas?

    Honestly enough they should, there are pushes to do so, it's just that funding is an issue, as always. Funny thing is, though, it's supposed to actually save money!

    Okay, up here in Alaska low oil prices have resulted in a government crisis - the government's income has dropped drastically. One of the reforms being put in place, finally, is 'community corrections', which has been shown to save money by actually preventing repeat criminal occurrences. As they mentioned, locking people up for long periods actually increases the chance they'll reoffend, especially if you don't provide support after they get out.

    So rather than locking somebody up for 12 years, you lock them up for, say, 4. You take the money for the next 4 years of prison and put it into rehabilitating the criminal, which is enough to cover extended services in prison, as well as at least 4 years outside, because, surprise, it's cheaper than keeping them in prison. The last 4 years of prison sentence avoided is pure savings, though they mentioned that they're putting half the money into what you mentioned - addressing the core situations causing criminality in the first place.

    As for the op -

    His attorneys claim that the code is "full of holes," including secret criteria and generic decisions that aren't as individually tailored as they have to be. For instance, they'll skew predictions based on your gender or age -- how does that reflect the actual offender...?

    Secret criteria is a problem, but gender and age have clear differences on how likely you are to re-offend. If you're a first time offender at 40 odds are something very strange happened, and if you fix that problem the person is unlikely to offend again, while being a first time offender at 15 is a bad sign that it might become a habit.

    --
    I don't read AC A human right
  9. Here is the problem by aepervius · · Score: 2

    Longer sentence increase the chance you re-offend, BUT often male are for a similarly violent offense given historically a longer or harsher sentence than female. So if the software use historical data to determine male should get a longer sentence , then it is a self fulfilling prediction.

    An algorithm should NEVER give a sentence length based on recidivism rate, it should solely based on the crime itself, and past recidivism. Giving longer sentence based on other criteria like probability f recidivism is pre-crime sentencing and utterly disgusting.

    --
    C. Sagan : A demon haunted world:
    http://www.amazon.com/gp/product/0345409469/
    visit randi.org
  10. what I found in their source code: by BlytheBowman · · Score: 2

    Whats in the algorithim: if $offender = poor { $sentence = $sentence + 5 } if $offender = homeless { $sentence = $sentence + 10 } if $offender = latino { $sentence = $sentence * 2 } elseif $offender = black { $sentence = $sentence * 3 }

  11. Re:Duh... by Cederic · · Score: 2

    Actual repeat offender? Sure take that into account.

    Projected repeat offender on grounds of 'young man' ? Sorry but that's ageist and sexist, and very much grounds for challenge as basing sentencing on that runs very counter to provision of equal justice.

  12. Original article I read on the subject by JonahsDad · · Score: 3, Informative
    Living in Wisconsin, I remembered reading about this last month. Here's the article from the Milwaukee Journal Sentinel:

    http://www.jsonline.com/news/c...

    They rely on cookies to limit the number of articles that you read. Unlike similar sites, they don't block you if you block cookies.

  13. Re:And this is different how? by tsqr · · Score: 2

    He wasn't convicted of rape.

    Actually, the prescribed sentence for rape in California is 3, 6, or 8 years, depending on the circumstances; all of these are less than the maximum sentence for the crimes for which Brock was convicted. zhe was convicted on three felony counts: assault with intent to commit rape of an intoxicated or unconscious person, sexual penetration of an intoxicated person and sexual penetration of an unconscious person. Maximum prison sentence of 10 years.

    Hard to imagine that a computer would have come up with a sentence 1/20th of the maximum prison time.

  14. Re:Punishes the disadvantaged by JoeMerchant · · Score: 2

    As others have said, the humanitarian side of the system errs on the side of forgiveness. If someone's credit is tanked, then don't consider their credit rating as a positive indicator of their compatibility with societies rules and expectations, look for other positive signs. This isn't saying "bad credit score: throw them under the jail and let them rot" this is saying "good credit score: indication that this person makes good on their promises, on a first offense give them benefit of the doubt and simple probation"

    Unless you can show me some causal relationship between credit score and recidivism it should not be a consideration unless the case is about something related to personal finance. Even using it only as a positive has a skewing effect. Minorities, poor, elderly, young all are more likely to have poor credit ratings. So you are not helping the most disadvantaged groups but you are helping white, middle/upper class privileged people based on "evidence" that is unrelated to the case. It's no different than saying "he looks like a nice young man so we shouldn't punish him too hard". It makes zero sense.

    Y'all ever been to Texas? "he looks like a nice young man so we shouldn't punish him too hard" gets taken to the extreme of letting rich kids off lightly for manslaughter, and executing innocents. You can say "it makes zero sense," to which the Texas judge (and most of the community) will say "that's how we do things here, son."

    Algorithmic sentencing is one way to move toward a more consistent system, less subject to the vagaries of individual jurisdictions. Judgements still have to be made based on something, and credit scores (and, more specifically, the underlying data from which they are computed) are one of the strongest windows into personality and prediction of future behavior we have in today's society. Should it be different? Maybe, but I'd rather not have to submit to monthly personality interviews with my state sponsored psychologist - especially if I haven't been convicted of anything yet.

  15. Algorithmic sentencing is a terrible idea by sjbe · · Score: 3, Insightful

    Y'all ever been to Texas?

    Not only have I been there I have family that lives there. Outside of Austin they should ask for a passport for most of the state to go there. I make no apologies when I say many Texans have some seriously messed up ideas about what constitutes "justice".

    Algorithmic sentencing is one way to move toward a more consistent system, less subject to the vagaries of individual jurisdictions.

    Consistency isn't necessarily as valuable as you might be implying. The entire point of having a judge is, you know, to judge things and come to a reasoned opinion about how the law should apply to a particular case. I could see an algorithm being useful as an aid to advise a judge on possible options but there are WAY too many corner cases for it to be a good idea to put it front and center. Are judges imperfect? Yep! That's why we have appeals systems. But you literally cannot come up with an algorithm that will properly address all the corner cases. You are merely turning the programmer and lawmakers into the de-facto judge which is a terrible idea. See three strikes laws if you need an example of how stupid algorithmic sentencing can be.

    Judgements still have to be made based on something, and credit scores (and, more specifically, the underlying data from which they are computed) are one of the strongest windows into personality and prediction of future behavior we have in today's society.

    I reject your framing of the issue. You are presuming that credit scores have any meaningful relationship to criminality without presenting any actual evidence that such an assertion is backed by facts. You are extrapolating purchasing and financial management behavior to have some relationship to criminality without any basis. Even a correlation isn't adequate because there are all sorts of ridiculous correlations between completely unrelated things. You have to PROVE a causal relationship between an individual person's credit score and their likelihood to commit future crimes for your argument to have any basis at all. Good luck with that.