Slashdot Mirror

← Back to Stories (view on slashdot.org)

Police To Test App That Assesses Suspects (bbc.com)

Posted by msmash on from the there's-an-app-for-that dept.

An anonymous reader writes: Police in Durham are preparing to go live with an artificial intelligence (AI) system designed to help officers decide whether or not a suspect should be kept in custody, BBC is reporting. The system classifies suspects at a low, medium or high risk of offending and has been tested by the force. It has been trained on five years' of offending histories data. One expert said the tool could be useful, but the risk that it could skew decisions should be carefully assessed. Data for the Harm Assessment Risk Tool (Hart) was taken from Durham police records between 2008 and 2012. The system was then tested during 2013, and the results -- showing whether suspects did in fact offend or not -- were monitored over the following two years. Forecasts that a suspect was low risk turned out to be accurate 98% of the time, while forecasts that they were high risk were accurate 88% of the time.

92 comments

  1. face recognition by Anonymous Coward · · Score: 0, Funny

    "is he black?" If so guilty

    1. Re:face recognition by Hognoxious · · Score: 2

      It's not Durham NC, it's Durham UK. So it's more likely to be the other way round.

      --
      Confucius say, "Find worm in apple - bad. Find half a worm - worse."
    2. Re:face recognition by Anonymous Coward · · Score: 1

      How to avoid the grievance mongers; only apply it to whites.

    3. Re:face recognition by cayenne8 · · Score: 2, Insightful
      Actually that brings up a good point.

      If the AI starts to evaluate, based SOLEY on data, that a particular racial group *does* tend to re-offend more often and is hence a higher risk....do we as the public start to believe it, or do we say that AI, even though purely scientific and logic based, is not Politically Correct and must have some artificial weights put into the algorithm to keep it from finding that some racial or economic strata of folks are more high risk and should be kept in jail?

      --
      Light travels faster than sound. This is why some people appear bright until you hear them speak.........
    4. Re:face recognition by bv728 · · Score: 5, Insightful

      You're assuming the data is unbiased - one of the known issues with systems like this is that biased policing can lead to datasets that don't reflect reality. A common example is how every other piece of data we have suggests that all racial and economic groups use illegal drugs at similar levels, but when you feed drug arrest records into predictive systems, they tell you lower income minority neighborhoods should be targeted. Which, logically, results in an even higher rate of arrest in those neighborhoods, but means there's even fewer police resources looking at other areas, potentially depressing arrests. And when you feed THAT data back in, it reinforces the same patterns.

      Existing police biases generate the data you're feeding into the system. Bias in, Bias out. It's not like this is a new idea.

    5. Re:face recognition by cayenne8 · · Score: 1

      You're assuming the data is unbiased - one of the known issues with systems like this is that biased policing can lead to datasets that don't reflect reality. A common example is how every other piece of data we have suggests that all racial and economic groups use illegal drugs at similar levels, but when you feed drug arrest records into predictive systems, they tell you lower income minority neighborhoods should be targeted. Which, logically, results in an even higher rate of arrest in those neighborhoods, but means there's even fewer police resources looking at other areas, potentially depressing arrests. And when you feed THAT data back in, it reinforces the same patterns.

      Ok, then don't use it for victimless crimes.

      I don't care who uses what drug....if they wanna kill themselves, that's their option.

      But hard to bias against who is committing the most gun violence, home invasions, rapes, violent crimes that actually impact those of us in regular society.

      I think its pretty hard to bias those....so, why not let the results fall where they may, whatever they may be.

      --
      Light travels faster than sound. This is why some people appear bright until you hear them speak.........
    6. Re:face recognition by Shotgun · · Score: 1

      Not to throw you reasoning askew, but could this be because the drug use is ancillary. Consider that police may not get calls for fights, break-ins, etc in higher income neighborhoods, so they have fewer reason to be patrolling those areas. The residents of higher income neighborhoods would be less likely to parade around the neighborhood with their drugs. That is, while they are using the drugs, they are not making themselves a public nuisance, let alone a public safety issue.

      If the point of the police is to insure public safety, instead of enforcing arbitrary laws, if follows that the reinforcing data you point out would actually still work toward the ultimate goal.

      --
      Aah, change is good. -- Rafiki
      Yeah, but it ain't easy. -- Simba
    7. Re:face recognition by Anonymous Coward · · Score: 0

      The inaccuracy could be inserted prior to data evaluation, it is GIGO, as always. AI decides on basis of its handlers' judgement.
      See: Computer Power and Human Reason by prof. Joseph Weizenbaum

    8. Re:face recognition by mjwx · · Score: 2

      Actually that brings up a good point.

      If the AI starts to evaluate, based SOLEY on data, that a particular racial group *does* tend to re-offend more often and is hence a higher risk....do we as the public start to believe it, or do we say that AI, even though purely scientific and logic based, is not Politically Correct and must have some artificial weights put into the algorithm to keep it from finding that some racial or economic strata of folks are more high risk and should be kept in jail?

      Well artificial weights have to be incorporated into the program. We're not talking about strong AI here (Artificial General Intelligence) that can determine things for itself, it has to be given parameters. Race should not be one of those parameters as it's not actually a determiner. Poor white neighbourhoods have the same crime problem as poor black ones.... there's just a greater volume of poor blacks (same for Asians and Hispanics). The RWNJ bogeyman of "Political Correctness" doesn't enter into it, parameters should be selected based on science (criminology and psychology).

      In other words, if a program lets a repeat offender out because he's white, the program is broken.

      If we do develop AGI and it turns out to be racist, humanity will have far bigger problems than a few people wrongfully kept in lockup.

      --
      Calling someone a "hater" only means you can not rationally rebut their argument.
    9. Re:face recognition by cayenne8 · · Score: 1
      I'm saying.."What IF"...the AI as it goes along, does on its own "see" that a particular race *does* statistically present a greater threat to general public.

      With todays sensitivity towards political correctness, prove to be unacceptable, EVEN if the AI is looked at and shown to be un-biased and just comes with this on observation.

      I"m guessing they'll be using parameters like sex....if that is the case, why not race?

      Is there really a legitimate reason, if the play field starts 100% fair and observation points to race being a factor observed...that is shouldn't be considered in the decision making process for "risk" of re-offense in this case?

      --
      Light travels faster than sound. This is why some people appear bright until you hear them speak.........
    10. Re:face recognition by gnick · · Score: 1

      Race should not be one of those parameters as it's not actually a determiner. Poor white neighbourhoods have the same crime problem as poor black ones.... there's just a greater volume of poor blacks (same for Asians and Hispanics).

      If that's true, then the algorithm should assign race a weight of zero, assuming that the other relevant factors are used as parameters. Whether or not it should be included is a matter of political correctness, not racial bias.

      ...parameters should be selected based on science (criminology and psychology).

      Why limit your parameter choice to what's suggested by criminology & psychology? Wouldn't science dictate that you use all useful data?

      --
      He's getting rather old, but he's a good mouse.
    11. Re:face recognition by mrclevesque · · Score: 1

      "But hard to bias against who is committing the most gun violence, home invasions, rapes, violent crimes that actually impact those of us in regular society."

      Still biased. Your list is avoiding violence and crimes from the higher classes.

    12. Re:face recognition by mrclevesque · · Score: 1

      I think the point was that even when police catch upper classes with illegal drugs they tend to be much more lenient.

      Also of course, the bigger your property the less likely the police will become aware of a crime happening on it (and even if they do become aware they will tend to be more lenient).

    13. Re:face recognition by sheph · · Score: 0

      His list doesn't single anyone out by class, and he doesn't say how he intends to gage metrics. However, crime from the higher classes isn't as prevalent. Let's face it, when you've got something to lose you're a little bit more careful about what you do. When you've got the IDGAF attitude because "I'm poor anyway what ya gonna do put me in jail and feed me; give me a place to live?" then it's not as big of a deal to commit crime. I've seen this with my own eyes. That's not to say there aren't spoiled rich kids out there committing crime. There absolutely are, and because they can afford better representation to come up with nonsense like affluenza more of them get by with it. But it's still far less percentage wise.

      --
      I don't believe in karma, I just call it like I see it.
    14. Re:face recognition by Anonymous Coward · · Score: 0

      A common example is how every other piece of data we have suggests that all racial and economic groups use illegal drugs at similar levels...

      [citation needed]

      Here's one though: "Little consensus exists regarding the relationship between socioeconomic status (SES) and substance use."

    15. Re:face recognition by cayenne8 · · Score: 1

      Still biased. Your list is avoiding violence and crimes from the higher classes.

      Err...exactly how is that?

      Are you saying that in higher classes, if they find dead bodies, they don't investigate and don't find and try to convict people of murder?

      I would posit higher class violence crimes would be taken into consideration just like lower class violent crimes. I would guess, however, you'd see less of it in higher classes, but that's just purely my observation.

      --
      Light travels faster than sound. This is why some people appear bright until you hear them speak.........
    16. Re:face recognition by mrclevesque · · Score: 1

      "gun violence, home invasions, rapes, violent crimes"

      Home invasions, gun violence or violent crimes, and I'm guessing rapes, tend to be more common in lower classes, on the other hand higher class crimes tend to be more dependent on capital expenditures.

      "If the AI starts to evaluate, based SOLEY on data"

      And the legal system is more lenient on higher classes, in general even for non victimless crimes. In other words the data is systemically biased, so the evaluation will be too.

    17. Re:face recognition by mrclevesque · · Score: 1

      See my reply here:

      https://slashdot.org/comments....

  2. So... by freeze128 · · Score: 2

    The cops aren't doing the profiling, the app is. Nice.

    1. Re:So... by Graydyn+Young · · Score: 1

      Done properly, this could be used as a way to prevent profiling. An algo can only make decisions based on the data provided to it. If race isn't provided as an input, it won't affect the decision. Humans can't make the same claim, as prejudices can sneak into our decisions unconsciously.

    2. Re:So... by ToTheStars · · Score: 1

      But the training data came from human judgment, in which case the algorithm has almost certainly inherited whatever biases were in that data.

    3. Re:So... by speedplane · · Score: 4, Interesting

      Done properly, this could be used as a way to prevent profiling. An algo can only make decisions based on the data provided to it. If race isn't provided as an input, it won't affect the decision. Humans can't make the same claim, as prejudices can sneak into our decisions unconsciously.

      There are so many ways that the algorithm can introduce bias, even if race isn't provided as an input, if other factors that may be highly correlated by race are (e.g., home zip code, occupation, income, etc.).

      --
      Fast Federal Court and I.T.C. updates
    4. Re:So... by Shotgun · · Score: 1

      So, continue to train it. With proper feedback factors, the bias should lose influence on the outcome. If it doesn't, it isn't a very good AI.

      --
      Aah, change is good. -- Rafiki
      Yeah, but it ain't easy. -- Simba
    5. Re:So... by Anonymous Coward · · Score: 0

      The cops aren't doing the profiling, the app is. Nice.

      Since the cops don't have a job that requires thought anymore, we can replace trigger-happy racist meatsacks with bots that tend not to shoot (black people) first and ask questions later. Nice.

    6. Re:So... by StikyPad · · Score: 1

      The problem is that the ML was trained on existing data, which is itself based on human and systemic biases, which means it's going to reinforce those biases. If someone is more closely watched because of some characteristic (age, gender, race, zip code, online habits), they are more likely to be caught for the same crime as someone who is not watched as closely. It *might* be possible to control for this, but it should at least be identified as a risk.

      --
      https://www.eff.org/https-everywhere
    7. Re:So... by Anonymous Coward · · Score: 0

      TFS says the data was verified over two years (i.e. at time of arrest app said something, but they didn't use that data to make a decision) and it was already quite accurate. That should mean its training is quite good so far. If most reoffenders are black, then bingo, you've identified a societal problem affecting the black community and society should figure out a way to fix it.

  3. So will it be more accurite than a human? by maas15 · · Score: 2

    Wonderful, more people get to find out that neural networks are a great way of coming to the same conclusion that any normal adult human could have come up with - after being woken up in the middle of the night after a evening of hard drinking.

  4. So they have a flow chart? by Anonymous Coward · · Score: 0

    White = let go.
    Non white = keep in custody.

    1. Re:So they have a flow chart? by Anonymous Coward · · Score: 0

      White = let go.
      Non white = keep in custody.

      Thus 98% accuracy.

    2. Re:So they have a flow chart? by Anonymous Coward · · Score: 0

      This is the UK, so it's the other way around. They don't go after organized child rapists if they're non-white because they're too afraid of being called racist.

    3. Re:So they have a flow chart? by sheph · · Score: 0

      There are areas in the US that are this way as well. Particularly after the whole BLM movement. In Chicago crime has skyrocketed because the police have pulled back from certain areas and won't engage with criminals.

      --
      I don't believe in karma, I just call it like I see it.
  5. And Demolition Man is prophetic again... by the_skywise · · Score: 4, Interesting

    https://www.youtube.com/watch?...

    (Okay it's not quite AI assessing of a subject but can this type of AI assist be far behind?)

    1. Re:And Demolition Man is prophetic again... by Anonymous Coward · · Score: 0

      SECRETARY TILLERSON: I want to welcome Foreign Minister Lavrov to the State Department and express my appreciation for him making the trip to Washington so that we can continue our dialogue and our exchange of views that began in Moscow with the dialogue he hosted on a very broad range of topics. Thank you.

      QUESTION: Does the Comey firing cast a shadow of your talks, gentlemen?

      FOREIGN MINISTER LAVROV: Was he fired?

      QUESTION: Yes.

      FOREIGN MINISTER LAVROV: You are kidding. You are kidding.

      Washington, DC May 10, 2017

    2. Re:And Demolition Man is prophetic again... by GameboyRMH · · Score: 1

      This is more similar to the Sybil system in Psycho-Pass. Well, what the Sybil system is *supposed* to be anyway...

      --
      "When information is power, privacy is freedom" - Jah-Wren Ryel
  6. Is he guilty, let's find out (wait for ads first) by TimothyHollins · · Score: 1

    Are we gonna replace Judge Judy with an app?

    This might be a non sequiteur, but I'd love a gps with Judhe Judy's mildly irritated voice.

  7. 98% accurate? by Calydor · · Score: 1

    It almost sounds to me like it was 100% accurate - if 2% of those deemed low risk of offending again then went and broke the law again, isn't that ... well, the definition of low risk? Same for the high risk result - if I understand it correctly, only 12% of those didn't break the law again.

    --
    -=This sig has nothing to do with my comment. Move along now=-
  8. Re:Is he guilty, let's find out (wait for ads firs by the_skywise · · Score: 1

    They've got Mr. T on Waze right now.

    "Turn left FOO!"
    (really)

    I could see Judge Judy next.

  9. Minority Report by U8MyData · · Score: 2

    Here we go. Easy is a four letter word folks. Dollar signs once again outweigh society. I'm glad I am past my prime and worry for the kids left behind.

    1. Re:Minority Report by Migraineman · · Score: 1

      Yeah, that was my first thought too.

  10. Hello World by wjcofkc · · Score: 1

    "Hi god. I'll be good, I promise."

    "I'll be the judge of that!"

    Meanwhile most of us are focused on the war with Oceania while more of this type of stuff comes into being.

    --
    Brought to you by Carl's Junior.
  11. Such a system doesn't have to perform well. by hey! · · Score: 2

    It just has to outperform cops.

    --
    Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    1. Re:Such a system doesn't have to perform well. by Anonymous Coward · · Score: 0

      That doesn't sound too difficult. Did they try using a dice instead of cops?

    2. Re:Such a system doesn't have to perform well. by Anonymous Coward · · Score: 1

      This. Even if the performance numbers look good. The cited statistics were for the edge cases, which I'm guessing are the easiest to classify. It only has value if it does better than the professional judgement of the officer who makes the decision normally. And of course, this doesn't even touch on the possibility for systematic discrimination, which would need it's own careful study and monitoring.

    3. Re:Such a system doesn't have to perform well. by Sparowl · · Score: 1

      We aren't setting the bar very high anymore for success, are we?

  12. Who gets the blame? by petes_PoV · · Score: 2

    So if (or when) this tool "decides" it is safe to release a suspect, who then goes on to commit another crime after release, who is reprimanded? who carries the can? who pays?

    Ultimately the responsibility still lies with the police force. It is their tool, the public safety is their responsibility.There needs to be reinforcement of this at every level, so that nobody can shrug their shoulders and say "the computer said it was OK".

    --
    politicians are like babies' nappies: they should both be changed regularly and for the same reasons
    1. Re: Who gets the blame? by gweihir · · Score: 1

      Indeed. However police is not held responsible for their mistakes more often than not, so "the computer decided it" is exactly what is going to happen.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    2. Re: Who gets the blame? by Joe_Dragon · · Score: 1

      and when the defendant requests the source code / logs for the system? They may need to give that out even if there contract / EULA says no.

      Court have Ordered Release of DUI 'Breathalyzer' Source Codes

    3. Re: Who gets the blame? by Anonymous Coward · · Score: 0

      if (skin.darkness > 0.6)
      {
                printf("he a good boy and he dindu nuffin you mean old cop pig!");
      }

    4. Re: Who gets the blame? by gweihir · · Score: 1

      Well, maybe. Still would not make much difference, as the sources and parametrization is pretty useless without the training data-set.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    5. Re: Who gets the blame? by Anonymous Coward · · Score: 0

      Tell him to call my law firm, Dewey Cheat'um and Howe.

      THey will get Mr. Dindu off.

    6. Re: Who gets the blame? by Ichijo · · Score: 1

      It might be better to let insurance companies decide whether to release a suspect, and to take the financial risk of doing so, and to get rewarded when the released suspect doesn't commit another crime.

      --
      Any sufficiently unpopular but cohesive argument is indistinguishable from trolling.
    7. Re: Who gets the blame? by ghoul · · Score: 3, Informative

      This would only be valid if Police was actually punished for their mistakes. Most often than not they are not so any computer system which can reduce mistakes even if it is punished for its new mistakes is an improvement.

      --
      **Life is too short to be serious**
    8. Re: Who gets the blame? by Anonymous Coward · · Score: 0

      Unless they release all of that data, then any judgement based on it is immediately rendered invalid. As the chain of evidence and it's processing cannot be verified to be correct.

      Of course actual justice means little to those who want a case off of their desk and absolutely nothing to those who want revenge, profit, or who scream "DO SOMETHING!" at the top of their lungs.

      So your statement is correct...

    9. Re: Who gets the blame? by gweihir · · Score: 1

      Just my thought.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  13. Counter-app by Errol+backfiring · · Score: 3, Insightful

    Does this mean it would be possible to write a counter-app? I mean an app that tells you what to wear, what to say and how to behave such that the police app will judge you as "low risk"?

    --
    Nae king! Nae laird! Nae yurrupiean pressedent! We willna be fooled again!
    1. Re:Counter-app by Anonymous Coward · · Score: 0

      It sounds like this is mostly using their criminal records to decide, so your counter-app would either suggest not committing crimes, or not getting arrested.

    2. Re:Counter-app by AmiMoJo · · Score: 2

      Unfortunately the top two things to avoid are being poor and being black.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    3. Re:Counter-app by Anonymous Coward · · Score: 0

      I'll write an app that does that. When you open it, it'll just say one thing:

      "Be a white guy."

    4. Re:Counter-app by tquasar · · Score: 1

      Dress like Pee-Wee Herman.

    5. Re:Counter-app by Shotgun · · Score: 1

      Been mostly done already. Watch this Chris Rock video.

      https://www.youtube.com/watch?v=uj0mtxXEGE8

      --
      Aah, change is good. -- Rafiki
      Yeah, but it ain't easy. -- Simba
    6. Re:Counter-app by Anonymous Coward · · Score: 0

      Heaven forbid but it ALREADY EXISTS:

          http://b96.cbslocal.com/2017/04/27/never-know-what-to-wear-amazons-new-camera-will-help-you-out/

      (Amazon's echo look camera, hopefully we let it pass us by...)

  14. What happens when it detects a black person? by Anonymous Coward · · Score: 0

    nt

    1. Re:What happens when it detects a black person? by Anonymous Coward · · Score: 0

      They get maced and tazed, immediately.

    2. Re:What happens when it detects a black person? by Anonymous Coward · · Score: 0

      or shot in teh face

  15. Not sure by kilodelta · · Score: 2

    How UK police work but in the U.S. all the cops have is charge data.

  16. 88% by PPH · · Score: 2

    So 12% that were judged high risk were in fact not.

    "It is better 100 guilty Persons should escape than that one innocent Person should suffer." - Benjamin Franklin

    Of course, this is the British we are talking about now. Blackstone put that error rate at around 1 in 10. So this app is statistically close enough for them.

    --
    Have gnu, will travel.
    1. Re:88% by Anonymous Coward · · Score: 0

      Believe it or not, British policing has come a long way since the 1760s. "Policing" has been invented, for one thing.

      But this isn't about locking people up and throwing away the key, it's about deciding whether they should be held for a few hours and brought before a magistrate for formal arraignment. Nobody is going to be sentenced to anything on this basis. It's universally understood that this is already an error-prone process, that's why we have trials, and 12% doesn't sound particularly excessive to me.

  17. Guilty in Advance by Lucidus · · Score: 3, Interesting

    I thought the idea was to detain people if they had already committed a crime, so I'm a little disturbed at the idea of holding them because you think they are likely to offend in future. If we are going to change the way we do these things, we will need to revamp our entire legal system (which I think would be a terrible mistake).

    1. Re:Guilty in Advance by aicrules · · Score: 1

      Being held in custody on suspicion of a crime or when charged with a crime are both legal parts of due process already. This app is just meant to add an automated analysis of data to help make the decision on whether to keep them in custody the legal amount of time or release them early.

    2. Re:Guilty in Advance by geekmux · · Score: 1

      I thought the idea was to detain people if they had already committed a crime, so I'm a little disturbed at the idea of holding them because you think they are likely to offend in future. If we are going to change the way we do these things, we will need to revamp our entire legal system (which I think would be a terrible mistake).

      The terrible mistake would be assume we still have a legal system.

      We don't. We have a justice system. BIG difference.

    3. Re:Guilty in Advance by Sigma+7 · · Score: 1

      I thought the idea was to detain people if they had already committed a crime, so I'm a little disturbed at the idea of holding them because you think they are likely to offend in future.

      The legal systems (at least the western ones) already try to determine whether or not someone is going to reoffend. There's basically two groups - ones with a single infraction, and ones who chronically violate the law.

      Since one of the objectives is to reduce recitivism (or at least the first offense), that's why there's social programs. Make sure people are kept fed, happy and safe, along with giving them the tools necessary to become productive.

      If we are going to change the way we do these things, we will need to revamp our entire legal system (which I think would be a terrible mistake).

      The only change needed is to make everything accurate. Ideally you want 100% accuracy, but I'm willing to settle if there's a false positive rate of (1/10)^20 - a one in a billion chance of one of the billion inhabitants of earth being falsely accused.

      By the way - Canada implemented a Dangerous Offenders Act. If an individual is established to have a pattern of violent criminal activity, that's grounds for indefinite detention. Generally, someone targeted by that act is no longer a risk of re-offending because it is guaranteed to happen.

    4. Re:Guilty in Advance by Anonymous Coward · · Score: 0

      The terrible mistake would be assume we still have a justice system.

      We don't. We have a legal system. BIG difference.

      FTFY.

      Justice has little to do with Legality, and more to do with Morality / Ethics. Just because it's legal for me to poison your water supply, doesn't mean it's justifiable to you.

      So, the GP was correct. It's a legal system that is increasingly desiring the ability to remove "inconvenient" or "undesirable" individuals at it's whim, and making every effort to make that desire a reality.

      BTW, There is no such thing as "precrime", those who practice it have merely redefined "crime" to be "anything the powers that be disagree with".

    5. Re:Guilty in Advance by Anonymous Coward · · Score: 0

      SO? The ONLY thing that should matter in such a case is any evidence you have that the person being held/suspected actually committed the crime they are suspected of committing NOT whether the person MAY commit a completely different/unrelated crime. A Judge may keep someone locked up 'indefinitely' (pending trial, through a trial etc.) if the evidence is strongly against that person OR if it's not so strong but the person has threatened someone in connection with the original crime...e.g. I'm thinking here in terms of say murdering someone & then threatening the life of someone else connected to the first event...but in all cases you still have to have evidence that these things occurred. You can't use a statistical model that says something to the effect of 'this person has committed crimes in the past & is likely to commit an entirely different crime in the future'...so WHAT? That has no bearing on an individual's right to due process & freedom.

      NO amount of 'guess work' about someone's POTENTIAL future actions should ever be used to incarcerate someone & remove a fundamental liberty. In my world I call that a 'police state'/dictatorship etc.

      The only potentially useful benefit of this is in terms of deciding on parole for an already convicted felon. Since parole is about 'cutting a criminal some slack' not about punishing someone further for something they haven't done then it may be valid to use this app in such cases.

    6. Re:Guilty in Advance by Anonymous Coward · · Score: 0

      Sorry but you're talking about 2 ENTIRELY different things.

      1) Determining if someone already convicted of a crime should be let go, e.g. given parole vs
      2) Locking up someone simply suspected of a crime or even charged with a crime but not convicted for something entirely unrelated you THINK they may do in the future.

      If you seriously believe that even a 'one in a billion' false positive rate in the latter case is justified then its time for the revolution to start. If you have insufficient evidence against an individual to keep them locked up either as a 'suspect' (so a short period of time while you gather more evidence to charge them) or to convict them now that you've charged them with a crime (so keeping them locked up until the trial is over) then you have NO reason to keep someone locked up & remove a fundamental human right. This isn't just a 'slippery slope' this is full on 'jump off a tall building without a safety net', society will truly given up their freedoms for the sake of 'safety' once again & the lessons of the past will have be rightly & truly forgotten. Absolutely amazing.

      Note, the Canadian Dangerous Offenders Act can only be used against someone actually convicted of a crime & the designation MUST be requested to be applied at the time of sentencing NOT retroactively after you've sentenced someone and then decide at the time of their release you don't really want to let them go. As it applies to this discussion you don't need a freakin' APP to tell you someone is a 'repeat violent offender', you've already convicted them a bunch of times for the same/similar act. If you haven't then there is NO 'pattern of violent criminal activity', and when applied to say serial killers/serial rapists where you just convicted them of multiple counts of crimes then simply give them 'life in prison with no possibility of parole' & be done with it. It is 'lazy legislation' but even so you don't need an app to tell you if a person should be sentenced to life in prison.

    7. Re:Guilty in Advance by Lucidus · · Score: 1

      The things you mention influence the duration of sentencing after a conviction, which is very different from what I understood this program to do; i.e., to determine whether (or how long) to detain suspects who have not yet been convicted of anything. I understand that the police have (and must have) some discretion in how they handle different individuals, but this still scares me.

    8. Re:Guilty in Advance by aicrules · · Score: 1

      No...they can arrest someone and immediately release them if they think they aren't a risk. While it varies by state/country, it is common that you can be held for up to 24 hours on suspicion of a crime. Then based on the seriousness of the offense you may be held longer. For example, if someone is arrested for possession of marijuana, they likely will be allowed to walk, whereas if they are arrested for murder they likely will not. So yes, how they handle you between arrest and charges is very much dependent on what you did and what you are likely to do.

  18. Evidence of the crime not withstanding? by Anonymous Coward · · Score: 0

    So, we're going to rely on an 'app' to make decisions about holding someone who hasn't actually been charged with a crime or maybe 1 who has but has not been convicted on an 'app'? Seriously? AND the app is supposedly answering a question that has no legal support for holding someone e.g. 'are they a risk for committing a FUTURE crime?'...you can't hold someone against their will simply because you suspect hey MAY re-offend (assuming they even committed a crime to begin with).

    The ONLY evidence that should be used to take away someone's freedom without a conviction is 'preponderance of the evidence in the crime they are suspected of committing'...I don't care if an app or a human makes the decision, you can't lawfully (or morally) hold someone against their will for something they MAY do. And even holding someone against their will for something you SUSPECT them of doing should require a fairly high 'burden of proof' (though I'd agree it doesn't have to be as high as evidence needed to convict).

    Has the world gone 'bat shit crazy' or something?

  19. Artificial by Anonymous Coward · · Score: 0

    Worst idea in the history of law enforcement. Brace for the lawsuit deluge.

  20. No, that will only reinforce bias by DoctorNathaniel · · Score: 1

    Any classification system requires unbiased true input to train against.

    Here's the model: Black people commit offenses and get arrested; white people are sometimes arrested but are much more likely to be let off with a warning. If the system has any proxy for 'black' in it's inputs, it will train on that. And as we know, there can be MANY such proxies.

    Retraining doesn't help: it makes the same judgement call as the officers, and there's not unbiased sample to test against.

    Math can't get you away from bias unless you can test the bias.

    1. Re:No, that will only reinforce bias by gnick · · Score: 1

      Black people commit offenses and get arrested; white people are sometimes arrested but are much more likely to be let off with a warning.

      If that's true, then the algorithm would correctly use race to determine whether an offender will be re-arrested. That doesn't make it "right," but it'll give the right answer to the question, "Will this person be arrested again?"

      --
      He's getting rather old, but he's a good mouse.
  21. Not cool by diesalesmandie · · Score: 1

    This sounds like one of those bullshit psychometric job interview tests. Its funny how society offloads the burden of reasoning onto popcorn science (psychometric job interview tests and this app for example) but ignores more rigorous science like what is happening to our planet (not perfect, and somewhat still subjective, but the majority of non-shill scientists agree). We are doomed as a species, maybe not in our lifetime but somewhere down the line for sure.

    --
    This is my sig, there are many like it but this one is mine
  22. What could POSSIBLY go wrong? by Rick+Schumann · · Score: 2

    Well, I see things are developing about how I expected: not only is so-called 'AI' making us lazy and less skilled, now it's going to make us dumber and less capable of thinking for ourselves. I see nothing good coming from this, it'll let dangerous people go and lock up people who don't need to be locked up. What's next, some hackneyed 'AI app' to decide whether to shoot someone or not? GTFO with this nonsense.

  23. "Computer says no" by whoever57 · · Score: 1

    https://www.youtube.com/watch?...

    --
    The real "Libtards" are the Libertarians!
  24. Re: Is he guilty, let's find out (wait for ads fir by Anonymous Coward · · Score: 0

    I was told Foo != Bar.

  25. Black by Anonymous Coward · · Score: 0

    What happens when you're black?

  26. AI is racist... by Anonymous Coward · · Score: 0

    ...sexist, xenophobic, and really really stupid. Way to go Durham police!

  27. Heh... by XSportSeeker · · Score: 2

    Precog system came earlier than I expected...

  28. The remedy by Anonymous Coward · · Score: 0

    Having nothing to do with or knowledge of any crime, I exercise my right to remain silent. This is the first thing out of a wise mans mouth when cops try to interrogate you.

  29. Covered nicely on SMBC by Anonymous Coward · · Score: 0

    Clicking through random SMBC and came across this one. Is obligatory SMBC a thing?

    http://www.smbc-comics.com/comic/minority-report

  30. No warrant, no detainment. by Anonymous Coward · · Score: 0

    No exceptions.

  31. just. no. by Anonymous Coward · · Score: 0

    its. not. fucking. AI. jesus.

  32. This is not new by Anonymous Coward · · Score: 0

    As someone who has worked in probation in the UK, I can tell you the only new thing about this is calling it an "app".

    For many years it has been routine to compete risk assessment forms based on offending history which produce a percentage risk of reoffending (e.g. OGRs).

    Sure, previously it was a form you'd complete on a PC in a browser, but if everything that has moved from a website to an app is news then I've got a hot story for you about Facebook...