Academics Confirm Major Predictive Policing Algorithm Is Fundamentally Flawed (vice.com)

An anonymous reader quotes a report from Motherboard: Last week, Motherboard published an investigation which revealed that law enforcement agencies around the country are using PredPol -- a predictive policing software that once cited the controversial, unproven "broken windows" policing theory as a part of its best practices. Our report showed that local police in Kansas, Washington, South Carolina, California, Georgia, Utah, and Michigan are using or have used the software. In a 2014 presentation to police departments obtained by Motherboard, the company says that the software is "based on nearly seven years of detailed academic research into the causes of crime pattern formation the mathematics looks complicated -- and it is complicated for normal mortal humans -- but the behaviors upon which the math is based are very understandable."

The company says those behaviors are "repeat victimization" of an address, "near-repeat victimization" (the proximity of other addresses to previously reported crimes), and "local search" (criminals are likely to commit crimes near their homes or near other crimes they've committed, PredPol says.) But academics Motherboard spoke to say that the mathematical theory that is used to power PredPol is flawed, and that its algorithm -- at least as pitched to police -- is far too simplistic to actually predict crime. Kristian Lum, who co-wrote a 2016 paper that tested the algorithmic mechanisms of PredPol with real crime data, told Motherboard in a phone call that although PredPol is powered by complicated-looking mathematical formulas, its actual function can be summarized as a moving average -- or an average of subsets within a data set. "The academic foundation for PredPol's software takes a statistical modeling method used to predict earthquakes and apply it to crime," reports Motherboard. "Much like how earthquakes are likely to appear in similar places, the papers argue, crimes are also likely to occur in similar places. Suresh Venkatasubramanian, a professor of computing at the University of Utah and a member of the board of directors for ACLU Utah, told Motherboard that earthquake data and crime data are, naturally, collected in different ways."

"I would say in our mind, the key difference is that in earthquake models, you have seismographs everywhere -- wherever an earthquake happens, you'll find it," Venkatasubramanian said. "The crux of the issue really is that to what extent are you able to get data about what you're observing that is not also totally on the model itself." "If you build predictive policing, you are essentially sending police to certain neighborhoods based on what what they told you -- but that also means you're not sending police to other neighborhoods because the system didn't tell you to go there," Venkatasubramanian said. "If you assume that the data collection for your system is generated by police whom you sent to certain neighborhoods, then essentially your model is controlling the next round of data you get."

Comments

Re:A city

[JoshuaZ]

You are missing the central point. People agree that putting police where crimes are more likely isn't a bad idea. The problem is that if one some crimes are reported or noticed by police, then having police in a given area is a self-reinforcing observation where the more police in an area, the more likely one is to detect crimes there and so the more police one puts there, even if it means other areas aren't going to get enough police. This is reinforced further by the fact that cops often feel a pressure to either directly make minimum quotas (e.g. at least some number of arrests and tickets) or are subject to other pressures which can cause them to engage in enforcement actions of things which are not crimes or are questionably criminal (e.g. disturbing the peace). If this is enough of an observational bias is probably a difficult question, but the researchers discuss it in more detail and it is something that likely can't get resolve by a few non-experts simply having a few paragraph conversation on Slashdot.

Re:A city

[Entrope]

I guess you missed the part where the professor explained that we have essentially complete coverage for earthquake detection. We don't have that for crime, and Americans generally reject the level of surveillance (total) that would be necessary to detect all crimes. If you use a predictive model to focus resources, but that model is trained on previous detections, you need that history to be statistically unbiased. Otherwise the bias tends to perpetuate itself, which is why the guy from the ACLU is concerned.

algorithms

[argStyopa]

I have mixed feelings about this.
First, the idea that algorithms alone can 'predict' something as subjective, human, and impulse-based as crime is ridiculous, and (I believe) born of a Utopian idea that taking people out of the equation can somehow remove bias, racism, and subjectivity from the process leaving some sort of idealistically mechanical, sterile system. For anyone who's worked in policing, crime prevention, or law enforcement fields, this should be a staggeringly stupid idea. Who's writing such algorithms but other people? On top of that, I expect there are now encrusted layers of ideology, in which results that don't conform to some utopian ideal of demographics are claimed to be 'racist' and formulaically 'corrected' to suit political goals, regardless of the facts of reality.

OTOH there is ABUNDANT work that shows that recidivism, particularly in the worst crimes, is concentrated in a surprisingly small number of individuals. I worked for a police dept where the longest serving officers maintained that 80%+ of the crimes were committed by a handful of families in the 50,000+ person city.

https://www.politifact.com/tex... lists some examples:

University of Pennsylvania criminologist Marvin Wolfgang tracked nearly 10,000 boys born in 1945 and living in Philadelphia from age 10 through 17; they ultimately gauged how often each boy came in contact with police for an offense. One upshot: 627 boys, 6 percent of the group, each accounted for five or more offenses, according to police reports. Those boys, Wolfgang wrote, were collectively identified as responsible for 52 percent of all the offenses recorded in the study and, he said, about two-thirds of all violent crimes believed to have been committed by the juveniles. In Patrick-speak, Wolfgang found that 6 percent of juvenile boys accounted for about half of alleged juvenile crimes.

The follow-up study, presented in progress in 1982, tracked more than 28,000 boys and girls born in 1958 who lived in Philadelphia from age 10 through 17. Among males, the study found, 61 percent of reported offenses were committed by 1,030 "chronic recidivists," comprising 7 percent of males in the study. That is, 7 percent of the boys accounted for 61 percent of the juvenile offenses.

In 2014, Swedish researchers drawing on records accounting for the experiences of 2.5 million people born in that country from 1958 to 1980 reported that from 1973 to 2004, some 1 percent of the population accounted for 63 percent of all violent crime convictions.

So it's clear that if we could identify this small percent and aggressively police them, we could make a sizable impact on crime.

Re:A city

[ceoyoyo]

There are lots of murders and burglaries that aren't reported. People disappear in big cities, and unknown bodies are discovered.

This report (from 2009) shows burglary is among the highest reported crimes, at 54%. https://www150.statcan.gc.ca/n...