Slashdot Mirror
Academics Confirm Major Predictive Policing Algorithm Is Fundamentally Flawed (vice.com)
An anonymous reader quotes a report from Motherboard: Last week, Motherboard published an investigation which revealed that law enforcement agencies around the country are using PredPol -- a predictive policing software that once cited the controversial, unproven "broken windows" policing theory as a part of its best practices. Our report showed that local police in Kansas, Washington, South Carolina, California, Georgia, Utah, and Michigan are using or have used the software. In a 2014 presentation to police departments obtained by Motherboard, the company says that the software is "based on nearly seven years of detailed academic research into the causes of crime pattern formation the mathematics looks complicated -- and it is complicated for normal mortal humans -- but the behaviors upon which the math is based are very understandable."
The company says those behaviors are "repeat victimization" of an address, "near-repeat victimization" (the proximity of other addresses to previously reported crimes), and "local search" (criminals are likely to commit crimes near their homes or near other crimes they've committed, PredPol says.) But academics Motherboard spoke to say that the mathematical theory that is used to power PredPol is flawed, and that its algorithm -- at least as pitched to police -- is far too simplistic to actually predict crime. Kristian Lum, who co-wrote a 2016 paper that tested the algorithmic mechanisms of PredPol with real crime data, told Motherboard in a phone call that although PredPol is powered by complicated-looking mathematical formulas, its actual function can be summarized as a moving average -- or an average of subsets within a data set. "The academic foundation for PredPol's software takes a statistical modeling method used to predict earthquakes and apply it to crime," reports Motherboard. "Much like how earthquakes are likely to appear in similar places, the papers argue, crimes are also likely to occur in similar places. Suresh Venkatasubramanian, a professor of computing at the University of Utah and a member of the board of directors for ACLU Utah, told Motherboard that earthquake data and crime data are, naturally, collected in different ways."
"I would say in our mind, the key difference is that in earthquake models, you have seismographs everywhere -- wherever an earthquake happens, you'll find it," Venkatasubramanian said. "The crux of the issue really is that to what extent are you able to get data about what you're observing that is not also totally on the model itself." "If you build predictive policing, you are essentially sending police to certain neighborhoods based on what what they told you -- but that also means you're not sending police to other neighborhoods because the system didn't tell you to go there," Venkatasubramanian said. "If you assume that the data collection for your system is generated by police whom you sent to certain neighborhoods, then essentially your model is controlling the next round of data you get."
The company says those behaviors are "repeat victimization" of an address, "near-repeat victimization" (the proximity of other addresses to previously reported crimes), and "local search" (criminals are likely to commit crimes near their homes or near other crimes they've committed, PredPol says.) But academics Motherboard spoke to say that the mathematical theory that is used to power PredPol is flawed, and that its algorithm -- at least as pitched to police -- is far too simplistic to actually predict crime. Kristian Lum, who co-wrote a 2016 paper that tested the algorithmic mechanisms of PredPol with real crime data, told Motherboard in a phone call that although PredPol is powered by complicated-looking mathematical formulas, its actual function can be summarized as a moving average -- or an average of subsets within a data set. "The academic foundation for PredPol's software takes a statistical modeling method used to predict earthquakes and apply it to crime," reports Motherboard. "Much like how earthquakes are likely to appear in similar places, the papers argue, crimes are also likely to occur in similar places. Suresh Venkatasubramanian, a professor of computing at the University of Utah and a member of the board of directors for ACLU Utah, told Motherboard that earthquake data and crime data are, naturally, collected in different ways."
"I would say in our mind, the key difference is that in earthquake models, you have seismographs everywhere -- wherever an earthquake happens, you'll find it," Venkatasubramanian said. "The crux of the issue really is that to what extent are you able to get data about what you're observing that is not also totally on the model itself." "If you build predictive policing, you are essentially sending police to certain neighborhoods based on what what they told you -- but that also means you're not sending police to other neighborhoods because the system didn't tell you to go there," Venkatasubramanian said. "If you assume that the data collection for your system is generated by police whom you sent to certain neighborhoods, then essentially your model is controlling the next round of data you get."
You are missing the central point. People agree that putting police where crimes are more likely isn't a bad idea. The problem is that if one some crimes are reported or noticed by police, then having police in a given area is a self-reinforcing observation where the more police in an area, the more likely one is to detect crimes there and so the more police one puts there, even if it means other areas aren't going to get enough police. This is reinforced further by the fact that cops often feel a pressure to either directly make minimum quotas (e.g. at least some number of arrests and tickets) or are subject to other pressures which can cause them to engage in enforcement actions of things which are not crimes or are questionably criminal (e.g. disturbing the peace). If this is enough of an observational bias is probably a difficult question, but the researchers discuss it in more detail and it is something that likely can't get resolve by a few non-experts simply having a few paragraph conversation on Slashdot.
I guess you missed the part where the professor explained that we have essentially complete coverage for earthquake detection. We don't have that for crime, and Americans generally reject the level of surveillance (total) that would be necessary to detect all crimes. If you use a predictive model to focus resources, but that model is trained on previous detections, you need that history to be statistically unbiased. Otherwise the bias tends to perpetuate itself, which is why the guy from the ACLU is concerned.
The data fed to the model is not just from police sent to neighbourhoods the model sends them to as suggested. The data is gathered from _all_ police dispatches. So the suggestion that the model is reinforcing its own biases is wrong.
"Consensus" in science is _always_ a political construct.
I have mixed feelings about this.
First, the idea that algorithms alone can 'predict' something as subjective, human, and impulse-based as crime is ridiculous, and (I believe) born of a Utopian idea that taking people out of the equation can somehow remove bias, racism, and subjectivity from the process leaving some sort of idealistically mechanical, sterile system. For anyone who's worked in policing, crime prevention, or law enforcement fields, this should be a staggeringly stupid idea. Who's writing such algorithms but other people? On top of that, I expect there are now encrusted layers of ideology, in which results that don't conform to some utopian ideal of demographics are claimed to be 'racist' and formulaically 'corrected' to suit political goals, regardless of the facts of reality.
OTOH there is ABUNDANT work that shows that recidivism, particularly in the worst crimes, is concentrated in a surprisingly small number of individuals. I worked for a police dept where the longest serving officers maintained that 80%+ of the crimes were committed by a handful of families in the 50,000+ person city.
https://www.politifact.com/tex... lists some examples:
So it's clear that if we could identify this small percent and aggressively police them, we could make a sizable impact on crime.
-Styopa
a predictive policing software that once cited the controversial, unproven "broken windows" policing theory as a part of its best practices.
The only thing "controversial" about broken window theory was that it worked.
The fact that it worked really bothered a certain class of people (people who wanted crime to be the fault of something other than criminals).
That really makes no sense for most crimes. Look at murder or burglary: it doesn't matter if police are in the neighborhood "noticing crimes" or not, it is going to get reported equally theoretically. The only way that applies is for victimless crimes. Traffic violations aren't going to be reported unless a police notices it.
Yes we do. If there is a murder, burglary, mugging it is going to be "detected" and reported no matter where it occurs by the populace. The only crimes that won't get reported are minor violations (traffic, etc). Police rarely detect crimes - they respond to crimes after they happen.
So.. while not an academic, this is pretty close to my field of research. Looking at their model, I am not surprised they sold this product but deeply disappointed. This is the type of model that is REALLY easy to sell to people, both law enforcement and the military (our customer) are enamored with them for their near magic ability to 'predict' things. Only they don't, they tend to fail in unpredictable ways. They are not bad in multi-model systems where you take a dozen or so different systems built by different teams, run them in parallel, then have subject matter experts ponder the conflicting results. But actual police out of a single model? Madness... or hubris.. or stupidity... or simply being enamored with a slick sales pitch from 'one of your own' offering to solve problems in the way you want them solved.
Oddly enough, we actually DID do a LEO model years back, which was actually pretty effective, but it encouraged things like community outreach and police/citizen interaction which worked really well for officers on the ground but pissed off lawmakers and 'police unions', so it was largely dropped.
Which gets back to this story and one of the fundamental flaws in such attempts. The decision makers are not interested in solutions that make things better for high crime areas in the first place, the people in those areas are not part of their power block. They want solutions that 'sound right' to people who live elsewhere and confirm what they already believe. Which is exactly what models like this are good at producing. They are kinda like torture... useless for prediction or information gathering, but an excellent political tool for confirming the story your career depends on being 'true'.
Police typically respond to calls, not just cruise around and "spot" crime. The fact they're in a neighborhood cruising already does not increase the likelihood of a call coming in; it may shorten response time, but it does not cause the call in the first place. If anything, it would tend to depress illegal activity in that neighborhood...
Browsing at +1 - no ACs, I ignore their posts. So refreshing!
You make a good point. Some crimes are going to be reported regardless of the level of police presence. Murder leaves behind a body that needs cleaning up, and reporting a burglary to law enforcement is penultimate to reporting it to an insurance company.
Some crimes will actually decrease in the the presence of heightened, especially visible, police activity. Many of these crimes, like drug activity, would have gone unreported unless they were interdicted by law enforcement.
In the presence of those tasked with upholding the law, more arrests will occur, even if that means LEOs have to police less severe offenses... arrests that are going to be reported.
Happiness in intelligent people is the rarest thing I know.
Ernest Hemingway
Do you want to pay for their education?
Do you pay property taxes, or live in a building where the owner pays property taxes? Then you're paying for their education already.
Everyone knows about the "US spends more per capita on healthcare and we have crappy service!" concept, but how many know that we spend just about the most per student on education, and have terrible results?
Perhaps the issue isn't the amount of money spent in these areas, but the goals/intent of the Governments who are doing the spending (which is valid for both cases, as Government spending in the US - Federal and State - accounts for about 65% of all healthcare spending). They do not have success as their goal, but rather continuation of their own bureaucratic existence.
Browsing at +1 - no ACs, I ignore their posts. So refreshing!
I think the definitions of under/over-policing are too dependent on the number of police available.
We call an area "over policed" if it seems like there are too many police patrols based on the amount of crime, when it seems to actually be driven by a sense that there are not enough police available in higher crime areas because they are misallocated to low crime areas.
I wonder if these terms would somewhat melt away if there were just more police overall? I would argue that the use of patrol cars and radios have created a false economy that suggests we can get away with too few police because they can be "efficiently" routed to places where crime has been reported. Generally low crime areas become effectively under-policed themselves and then become more susceptible to crimes of opportunity like burglary.
This is exactly what happens in my part of the city where I live, especially once warm weather hits. The crime rate is very low generally, but there's a huge uptick in burglary during spring and summer. Police and civic officials say there's nothing that can be done, mostly due to a lack of resources. The counter-argument is that more police patrols would increase criminal risk and reduce opportunity.
If you consider a thought experiment where the amount of police cars is held very low and most patrolling would need to be done in a non-motorized fashion, you would need more overall police since you couldn't just send patrol cars in response to criminal activity and zero police presence wouldn't be an option, either. Low crime areas would wind up with less criminal opportunity due to a more regular and permanent police presence.
The other dynamic that seems to drive low policing levels in generally low crime areas is the perception that since most of the crimes that do occur are property crimes, they are a low priority because the residents are generally affluent and have insurance which more or less eliminates the "victim".
If property insurance were much more expensive, say most people had a $10,000 deductible or higher and thus were self-insured for all amounts below that, I think property crime would get more police engagement. Victims would be more or less permanently victimized by material losses, since they would be very expensive to replace.
Nearly all police actions other than traffic citations come from dispatches - someone calling in/reporting a crime. Location of the officer has nothing to do with that, other than potentially shortening response time.
Browsing at +1 - no ACs, I ignore their posts. So refreshing!
There are lots of murders and burglaries that aren't reported. People disappear in big cities, and unknown bodies are discovered.
This report (from 2009) shows burglary is among the highest reported crimes, at 54%. https://www150.statcan.gc.ca/n...
Murder yes you're right, unless the area is dealing with a high number of murders. See the case of NYC in the 1970's, 'warm bodies' on the streets made a significant difference in the span of a few years. Burglary you're wrong on, more police or more active patrols decrease the possibility of those types of crimes happening because the possibility of something happening in plain sight makes the individual reconsider their actions. See the "rational choice" theory out of criminology for example. It takes the belief that most people, knowing right from wrong will not take an action unless they fall into three basic groups. First being those who won't ever commit a crime, the second being those that will commit a crime if they know they won't be caught. The third being those who will commit a crime irregardless of circumstances, even if someone is standing over their shoulder. Depending on the studies, those numbers range from 30-40% who will never commit a crime. To the remaining 60% who might or will. Will generally making up 10-25% of that remainder depending on various other factors dealing from generational crime, to social influence.
CPTED(Crime prevention through environmental design) is the basis of reducing crime by deterring the actions of those who "might" and "will" commit a crime. Whether it be more patrols, building designs that don't leave dark areas, motion lights/cameras, and so on. It's also heavily used in internal theft-prevention in every business around the world because it works, and works well. You can turn bad areas that are effectively ghettos into crime free areas by increasing property values, bringing in businesses that employ, reducing petty crime and poor education and so-forth. Having programs like Neighborhood watch or COP(citizens on patrol), to have more eyes looking for crime problems. All of that falls into the CPTED models.
Om, nomnomnom...
that violent crime is less common then we like to think? Of course a small percentage is responsible for all violent crime. There isn't enough to go around.
Also, not sure about Sweden in the 80s but in America, even today, our prison system chews you up and spits out broken people. That's been equally well documented.
Finally pre-90s is a bad place to get crime statistics from. Lead in the air was pretty obviously creating unhinged people. Again, there's plenty of studies to back this up because it's the only thing that can explain the across the board drop in crime.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
that's corralling it. Stick with me on this, it's long.
I saw this in my old town (I live in an apartment full of Indian H1-bs now and while they do take my jerbs they're about the least criminal people on earth if you don't count them hacking my cheap router from time to time).
I used to live in a cheap, working class neighborhood. Up the street were crack houses galore. Never once did they bother me. Only trouble I ever had was from a loser friend who's girlfriend robbed me.
I used to wonder why and then I found out. "Broken Window" policing is actually "Busted Heads" policing. My bud lived in one of those crack house apartments following a bad divorce (only thing he could afford/get after getting his credit destroyed). Got robbed, they caught the guy when his apartment manager went into the apartment and recognized his stuff there. They released the guy a few days later and he was still living next door to my bud when he finally got his credit fixed and moved out.
That was OK because they'd kept the crime inside. Every now and then one of them would venture out of their little hell hole and rob a liquor store or something.
The cops would come down like a ton of bricks. Everyone got arrested. And since they all had at least some pot half were probably gonna do a year or two in the clink. Especially the Dads, who would take the rap for the pot so the mom could at least stay out of prison. And during the raid you better believe heads got busted like crazy.
This kind of shit is used to force the lower caste to stay in their lane. Keep their head down. It's the nastiest form of oppression possible. It lets you and me ignore the problem of widespread poverty because when the poor make trouble there's a cop there ready to beat them the fuck down and a private prison system happy to lock 'em up for 3-5 years.
This is also why the drug war hasn't ended. Locking up randos for minor drug crimes is how you keep those folks on edge.
Now, you might be thinking, so what? They're criminals anyway. That's all well and good, but think about it. When Capitalism goes south what's suppose to fix it? The Answer is that Mr Factory Owner won't let the country go to shit because he lives in it. But Mr Factory owner and even his middle class servants can use tricks like this to control the populace why bother? What's to stop him from letting everything go to shit except where he lives? This is where oligarchy comes from.
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
That's the problem; we're number 3 in the world in spending per student, and we're 17th out of 22 in the OECD in high school graduation rates.
We should pay more attention to Eisenhower's warning about the Government-Education complex, and how it's feeding off of the US taxpayer as much as the military-industrial complex. For example, the DOD budget in 2016 was $585 billion; Federal and State spending on K-12 education in 2016 was $620 billion. How many people would guess we spend more on education of K-12 than we do on the military?
When DOD cost overruns are found, there is much howling about cutting back to the DOD, increasing oversight, etc. When the educational system fails (as it does), there is handwringing and demand to spend even more. Somehow society has been conditioned to accept failure of the Government-Education complex and reward that failure with even more dollars and apologies.
Browsing at +1 - no ACs, I ignore their posts. So refreshing!
Crime happens everywhere but it is not always reported. The worst, most critical crime in terms of its overall effect upon the public is never subject to a police dispatch. It happens at 1600 Pennsylvania Avenue and environs in Washington DC. It happens at Wall Street. And at Caracas, Mexico City, Dubai, Tel Aviv, Moscow, and even Geneva. It happens in marble buildings and fancy restaurants and hotels and involves people wearing Armani suits and fake sun tans. These crimes involve investment schemes, trade in drugs & guns, overthrowing governments, usurious loans to third world countries, manipulation of elections, monetary gifts to dictators and warlords, currency manipulations, deforestation for private profit and more. These criminals enrich each other at the expense of billions who suffer from the ravages of war, hunger, disease and deprivation.
The police are not called because these perpetrators are above the law.
...omphaloskepsis often...
Except the ISSUE IS NOT REPORTING or "noticing crimes" - IT'S PREDICTING CRIMES that's the issue.
Algorithm doesn't notice nor report crimes. It predicts where to send the police.
Resulting in a "garbage in = garbage out" predictive result based on reinforcement of outdated data.
E.g. If there was an arrest of a guy selling pot in front of a local Starbucks last month, and another guy arrested selling meth in a parking lot of a mall - algorithm now dictates through "near-repeat victimization" that both the Starbucks and the mall AND EVERYTHING AROUND THEM are likely locations of future crimes.
And should cops actually notice something in that area aroooouuund the location of a previous crime while being under pressure to fulfill their monthly quotas - it is seen as a validation of the predictive powers of the magical AI.
Rinse and repeat.
It's "Round up the usual suspects!" - only with locations and "supported" by math.
Pretty soon you have cops policing parking lots for broken tail lights and ID-checking everyone around a Starbucks, falling number of arrests for preventable crimes (such as selling drugs or opportunistic crimes) - with actual number of crimes on the increase city-wide.
Cause everyone is listening to the magical algorithm, designed to predict earthquake aftershocks.
Instead of having police patrolling even there where no crimes are being reported - e.g. cause the locals don't trust the police or are afraid of reprisals from the drug dealer next door.
Mit der Dummheit kämpfen Götter selbst vergebens
No one realized that those parts of the airplanes weren't as important to protect because the planes still returned though they were hit there. Protecting the parts where the returning planes didn't get hit was much more important, as obviously, planes hit there never made it back.
This is called survivorship bias. And systems that try to predict crimes from past crime numbers suffer heavily from survivorship bias.
There are about 240 million 911 calls each year. There are about 900,000 sworn police officers nationwide. That's about 266 calls annually per police officer. Given there are about 240 work days per year (5 per week, 2 weeks vacation, 2 weeks of holidays), that's a little more than 1 call per day. There were about 10,550,000 arrests in 2017, or about 1 per month per police officer. So they're responding to about 22 more dispatches (from 911) than they are arresting. What are the odds your average police officer will only arrest things he sees happening on the street, and not any from the 22 dispatches per month?
Browsing at +1 - no ACs, I ignore their posts. So refreshing!