Academics Confirm Major Predictive Policing Algorithm Is Fundamentally Flawed (vice.com)

An anonymous reader quotes a report from Motherboard: Last week, Motherboard published an investigation which revealed that law enforcement agencies around the country are using PredPol -- a predictive policing software that once cited the controversial, unproven "broken windows" policing theory as a part of its best practices. Our report showed that local police in Kansas, Washington, South Carolina, California, Georgia, Utah, and Michigan are using or have used the software. In a 2014 presentation to police departments obtained by Motherboard, the company says that the software is "based on nearly seven years of detailed academic research into the causes of crime pattern formation the mathematics looks complicated -- and it is complicated for normal mortal humans -- but the behaviors upon which the math is based are very understandable."

The company says those behaviors are "repeat victimization" of an address, "near-repeat victimization" (the proximity of other addresses to previously reported crimes), and "local search" (criminals are likely to commit crimes near their homes or near other crimes they've committed, PredPol says.) But academics Motherboard spoke to say that the mathematical theory that is used to power PredPol is flawed, and that its algorithm -- at least as pitched to police -- is far too simplistic to actually predict crime. Kristian Lum, who co-wrote a 2016 paper that tested the algorithmic mechanisms of PredPol with real crime data, told Motherboard in a phone call that although PredPol is powered by complicated-looking mathematical formulas, its actual function can be summarized as a moving average -- or an average of subsets within a data set. "The academic foundation for PredPol's software takes a statistical modeling method used to predict earthquakes and apply it to crime," reports Motherboard. "Much like how earthquakes are likely to appear in similar places, the papers argue, crimes are also likely to occur in similar places. Suresh Venkatasubramanian, a professor of computing at the University of Utah and a member of the board of directors for ACLU Utah, told Motherboard that earthquake data and crime data are, naturally, collected in different ways."

"I would say in our mind, the key difference is that in earthquake models, you have seismographs everywhere -- wherever an earthquake happens, you'll find it," Venkatasubramanian said. "The crux of the issue really is that to what extent are you able to get data about what you're observing that is not also totally on the model itself." "If you build predictive policing, you are essentially sending police to certain neighborhoods based on what what they told you -- but that also means you're not sending police to other neighborhoods because the system didn't tell you to go there," Venkatasubramanian said. "If you assume that the data collection for your system is generated by police whom you sent to certain neighborhoods, then essentially your model is controlling the next round of data you get."

Comments

A city

[AHuxley]

Only has so many police to send to any neighborhood.
Have all the skilled police waiting around a low crime neighborhood is going to add to support times in crime filled neighborhoods.
The ability to pack a lot of police into the "neighborhood" that is full of criminals allows a rapid response to crime and criminals.
Call for help get paid police action.
More police are in the area to support the number of calls.
Don't have skilled police waiting around in the better low crime neighborhoods.
The "system" quickly works out what neighborhoods have the crime.
To map out the better neighborhoods with normal people who don't do crime all day and night.
The system is working perfectly and allows a city to place police jwre needed, in the most crime filled neighborhoods.

Re:A city

[JoshuaZ]

You are missing the central point. People agree that putting police where crimes are more likely isn't a bad idea. The problem is that if one some crimes are reported or noticed by police, then having police in a given area is a self-reinforcing observation where the more police in an area, the more likely one is to detect crimes there and so the more police one puts there, even if it means other areas aren't going to get enough police. This is reinforced further by the fact that cops often feel a pressure to either directly make minimum quotas (e.g. at least some number of arrests and tickets) or are subject to other pressures which can cause them to engage in enforcement actions of things which are not crimes or are questionably criminal (e.g. disturbing the peace). If this is enough of an observational bias is probably a difficult question, but the researchers discuss it in more detail and it is something that likely can't get resolve by a few non-experts simply having a few paragraph conversation on Slashdot.

Re:A city

[Entrope]

I guess you missed the part where the professor explained that we have essentially complete coverage for earthquake detection. We don't have that for crime, and Americans generally reject the level of surveillance (total) that would be necessary to detect all crimes. If you use a predictive model to focus resources, but that model is trained on previous detections, you need that history to be statistically unbiased. Otherwise the bias tends to perpetuate itself, which is why the guy from the ACLU is concerned.

Re:A city

[AHuxley]

Re "self-reinforcing observation"
No city has the extra police numbers to over police low crime areas and still have enough police for parts of the city filled with criminals.
Crime fills parts of US cities and over time its rather easy to map out, put up on a GUI map.
Every 911 call has to be responded to on time.
No city can just say that many of its police are kept on patrol in a very low crime area that day and the response time in a high crime area will take time.
Police do not want to see that their police support is kept far away from them in areas of a city with a constant crime problem.
The numbers of police, police patrols is not a large number.
Why would any city waste its limited numbers of police in low crime areas?
Thats real people in an area of the city with real crime that have to wait longer? Why do that?
he system exists to map and predict crime given past crime.
Fill that area with police and try and contain all the crime and criminals.

Re "areas aren't going to get enough police".
That would show on the reported maps and the system would report a need to place more police in that area as crime and criminals move into an area.
Insurance reports, reported crime, calls to 911, types of crime, arrests and resulting convictions would show a rising and changing crime rate.
Its not "questionably criminal" when lots of people go to court and then to prison from one area of a city a lot more than another.
Police could put on more patrols and try and get their ability time needed to get to a crime down.

Re:A city

[110010001000]

That really makes no sense for most crimes. Look at murder or burglary: it doesn't matter if police are in the neighborhood "noticing crimes" or not, it is going to get reported equally theoretically. The only way that applies is for victimless crimes. Traffic violations aren't going to be reported unless a police notices it.

Re:A city

[110010001000]

Yes we do. If there is a murder, burglary, mugging it is going to be "detected" and reported no matter where it occurs by the populace. The only crimes that won't get reported are minor violations (traffic, etc). Police rarely detect crimes - they respond to crimes after they happen.

Re:A city

[rmdingler]

You make a good point. Some crimes are going to be reported regardless of the level of police presence. Murder leaves behind a body that needs cleaning up, and reporting a burglary to law enforcement is penultimate to reporting it to an insurance company.

Some crimes will actually decrease in the the presence of heightened, especially visible, police activity. Many of these crimes, like drug activity, would have gone unreported unless they were interdicted by law enforcement.

In the presence of those tasked with upholding the law, more arrests will occur, even if that means LEOs have to police less severe offenses... arrests that are going to be reported.

Re:A city

[LostMyAccount]

I think the definitions of under/over-policing are too dependent on the number of police available.

We call an area "over policed" if it seems like there are too many police patrols based on the amount of crime, when it seems to actually be driven by a sense that there are not enough police available in higher crime areas because they are misallocated to low crime areas.

I wonder if these terms would somewhat melt away if there were just more police overall? I would argue that the use of patrol cars and radios have created a false economy that suggests we can get away with too few police because they can be "efficiently" routed to places where crime has been reported. Generally low crime areas become effectively under-policed themselves and then become more susceptible to crimes of opportunity like burglary.

This is exactly what happens in my part of the city where I live, especially once warm weather hits. The crime rate is very low generally, but there's a huge uptick in burglary during spring and summer. Police and civic officials say there's nothing that can be done, mostly due to a lack of resources. The counter-argument is that more police patrols would increase criminal risk and reduce opportunity.

If you consider a thought experiment where the amount of police cars is held very low and most patrolling would need to be done in a non-motorized fashion, you would need more overall police since you couldn't just send patrol cars in response to criminal activity and zero police presence wouldn't be an option, either. Low crime areas would wind up with less criminal opportunity due to a more regular and permanent police presence.

The other dynamic that seems to drive low policing levels in generally low crime areas is the perception that since most of the crimes that do occur are property crimes, they are a low priority because the residents are generally affluent and have insurance which more or less eliminates the "victim".

If property insurance were much more expensive, say most people had a $10,000 deductible or higher and thus were self-insured for all amounts below that, I think property crime would get more police engagement. Victims would be more or less permanently victimized by material losses, since they would be very expensive to replace.

Re:A city

[LynnwoodRooster]

Nearly all police actions other than traffic citations come from dispatches - someone calling in/reporting a crime. Location of the officer has nothing to do with that, other than potentially shortening response time.

Re:A city

[ceoyoyo]

There are lots of murders and burglaries that aren't reported. People disappear in big cities, and unknown bodies are discovered.

This report (from 2009) shows burglary is among the highest reported crimes, at 54%. https://www150.statcan.gc.ca/n...

Re:A city

[Mashiki]

Murder yes you're right, unless the area is dealing with a high number of murders. See the case of NYC in the 1970's, 'warm bodies' on the streets made a significant difference in the span of a few years. Burglary you're wrong on, more police or more active patrols decrease the possibility of those types of crimes happening because the possibility of something happening in plain sight makes the individual reconsider their actions. See the "rational choice" theory out of criminology for example. It takes the belief that most people, knowing right from wrong will not take an action unless they fall into three basic groups. First being those who won't ever commit a crime, the second being those that will commit a crime if they know they won't be caught. The third being those who will commit a crime irregardless of circumstances, even if someone is standing over their shoulder. Depending on the studies, those numbers range from 30-40% who will never commit a crime. To the remaining 60% who might or will. Will generally making up 10-25% of that remainder depending on various other factors dealing from generational crime, to social influence.

CPTED(Crime prevention through environmental design) is the basis of reducing crime by deterring the actions of those who "might" and "will" commit a crime. Whether it be more patrols, building designs that don't leave dark areas, motion lights/cameras, and so on. It's also heavily used in internal theft-prevention in every business around the world because it works, and works well. You can turn bad areas that are effectively ghettos into crime free areas by increasing property values, bringing in businesses that employ, reducing petty crime and poor education and so-forth. Having programs like Neighborhood watch or COP(citizens on patrol), to have more eyes looking for crime problems. All of that falls into the CPTED models.

Re:A city

[RazorSharp]

Nice poem.

Re:A city

[JoshuaZ]

Nearly all police actions other than traffic citations come from dispatches - someone calling in/reporting a crime.

Do you have citation or source for that claim?

Re:A city

[WhiplashII]

I think they are missing that crime is not reported by police. Crime is reported by crime victims.

Corollary: If you do not report a crime, the police will not work the crime. But I'd think that most people understand that issue.

Re:A city

[PPH]

People disappear in big cities, and unknown bodies are discovered.

And thus they are reported. They might not be solved, but a dead body discovered goes into the database 100% of the time. It might turn out to be a death due to natural causes, but nobody allows corpses to lie around stinking without telling someone.

I'd venture a guess that the most under reported crimes (particularly in light of your Canadian study) are failed assault/robbery attempts. Where the victim drives the attacker off, possibly with a weapon. The victim might end up in more trouble for possessing a knife (or, God forbid, a gun) and it's just not worth the trouble to call it in.

Re:A city

[Mashiki]

What you're really looking for in terms of solved crimes is what's called the clearance rate. Or the percentage of crimes reported that are solved. For example, in my town the assault 1(i.e. fist fighting) has a 95% clearance rate. The vandalism clearance rate is about 7%, it's hard to catch people in the act even these days with all the cameras around.

Depending on where you are in Canada, the most under reported crimes are theft under $5k and various victimless crimes like prostitution. The previous conservative government rescinded and rewrote large sections of the criminal code dealing with self-defense granting more leeway to an individual if they go over the proportional line of self-defense(i.e. they come at you with fists, you go to a bat isn't proportional). In turn you're now seeing more cases of "so-and-so person in the asshole of nowhere shoots and kills one person in a group of 5 people who were attempting to steal property/break into their home/etc" or "store owner locks three thieves into cargo van while awaiting police response" cases becoming more common.

This is especially true in the asshole of nowhere, where policing isn't minutes away it can be as long as a day before there's a response. 12hrs for example in rural Manitoba, Saskatchewan or Alberta isn't unheard of. Even down here in Southwestern Ontario, you can wait 4 hours for the OPP to show up after an assault. Mainly because, even if the constables are posted to your town they may have a patrol area 200km away from their base station.

Re:A city

[PPH]

What you're really looking for in terms of solved crimes is what's called the clearance rate. Or the percentage of crimes reported that are solved.

No. What I'm looking for is the actual crime rate. Statistics will steer different police departments to select the best tactics to improve the clearance rate on those crimes.

Canada ... granting more leeway to an individual if they go over the proportional line of self-defense

Good. Canada needs to stop coddling criminals.

Re:A city

[aaarrrgggh]

Because generally reporting a burglary doesn’t do any good. The only real reason to report is for insurance purposes.

Re:A city

[Sique]

Basicly it's the same argument that caused high loss of airplanes in World War II. Whenever the british airplanes returned to their airfields, engineers were analyzing the places where the airplanes were hit, and reinforcing that part of the airplane. Sadly, the numbers of lost planes didn't go down.

No one realized that those parts of the airplanes weren't as important to protect because the planes still returned though they were hit there. Protecting the parts where the returning planes didn't get hit was much more important, as obviously, planes hit there never made it back.

This is called survivorship bias. And systems that try to predict crimes from past crime numbers suffer heavily from survivorship bias.

Re:A city

[LynnwoodRooster]

There are about 240 million 911 calls each year. There are about 900,000 sworn police officers nationwide. That's about 266 calls annually per police officer. Given there are about 240 work days per year (5 per week, 2 weeks vacation, 2 weeks of holidays), that's a little more than 1 call per day. There were about 10,550,000 arrests in 2017, or about 1 per month per police officer. So they're responding to about 22 more dispatches (from 911) than they are arresting. What are the odds your average police officer will only arrest things he sees happening on the street, and not any from the 22 dispatches per month?

Re:A city

[JoshuaZ]

Thank you. That's a very reasonable back of the envelope estimate to support the claim in question.

Re:A city

[LynnwoodRooster]

You're welcome!

Re:A city

[sjames]

Possibly, but note well, divide a uniform crime area into sectors. Each sector has the same level and severity of crime. Flip a coin to decide which sectors get the most police presence. The more patrolled sectors will naturally be the ones where more criminals are caught, and so in the crime report will be "higher crime" areas. They will continue to get more patrols because they will continue to have higher crime stats.

That effect CAN be taken into account in the software, but it isn't.

Re:A city

[sjames]

You have managed to prove nothing. You haven't even established a probability of anything based on logic.

Re:A city

[sjames]

Crimes certainly are reported by police, in the form of arrests.

Cop sees a drug deal go down. Cop arrests drug dealer. It shows up in the crime stats.

I would guess that very few drug deals get reported by concerned citizens.

Re:A city

[LynnwoodRooster]

Funny, the person I was responding to sees it as a reasonable conjecture. What would be your reasoning that the one arrest per month per police officer is NOT the result of their daily dispatch call?

Re:A city

[_Sharp'r_]

Except of course that crime rates can be based on crimes reported by crime victims/observers and you can just throw out the reports directly from officer observations. No one bases crime rates on the number of criminals "caught", or where they are caught. The statistic used is where the crime reported occurred.

Re:A city

[AHuxley]

The FBI has been collecting great amounts of data on different crime all over the USA for many years.
The locations, distance calculations at a crime with the use of weapons, type of crime, who did the crime and their connection to that area.
The data exists for any part of the USA.
City and state police can now do the same "math" as the FBI did looking g back over years of complex collected crime data sets.
Place more police where crime and criminals are and the wider city is kept much more safe.
Crime and criminals are contained to set areas of a city where more police can respond.
Allowing more crime to spread out over a nice city is not a good idea given the years of data that shows where crime is.
When people call for the police the time needed is less.
When police need support more police are in the area to reduce the time needed to get support.
A city can only pay for so many police, their transport, their pensions, the hours they can work, over time.
Every use of police has to be considered for an area filled with crime and criminals.
No city can have the added cost of extra police driving around very low crime areas.
When 911 is called, people expect police to arrive soon.
Having extra police in another nice part of a city, in a low crime area is not going to get police that are needed to that crime quickly.

Re:A city

[sjames]

I said you have PROVEN nothing. Starting with 911 calls. People call 911 because of criminal activity, medical emergency, and fire. Also because McDonalds is out of Apple pies, Best Buy won't honor my fake coupon for 110% off, do you know what time it is, and sorry my butt dialed it. But you decided they're all criminal activity. Beyond that, I would find it more likely that a crime a cop actually witnesses is more likely to lead to an arrest than one that happened 30 minutes or more before a cop arrived on scene.

Given that, the rest is just speculation based on poorly quantified events.

Re:A city

[sjames]

That is at odds with TFA based on people who actually examined the use of the software.

You COULD throw out the reports from police, but they DON'T.

Police presence leads to predictable results

[rmdingler]

By targeting a portion of a city or neighborhood with more LEOs, you will create more arrests there.

There are undoubtedly several areas in virtually every city that are predisposed to criminal activity... areas that have a high concentration of poor people or others that are resigned to accept more overt peddling of the drug and skin trades, for instance. If you're looking to prove an algorithm's efficacy, deploying additional police personnel and assets to any of these areas will get it done.

Re:Police presence leads to predictable results

[Anonymous Coward]

By targeting a portion of a city or neighborhood with more LEOs, you will create more arrests there.

That only works if there's something to arrest people for.

There are undoubtedly several areas in virtually every city that are predisposed to criminal activity... areas that have a high concentration of poor people or others that are resigned to accept more overt peddling of the drug and skin trades, for instance..

So what? Abandon the people living there by letting criminals have free reign? I love that the "solution" to politically inconvenient crime all over the Western world is to simply stop policing said crime because it proves the "wrong" people right. Problem solved, once and for all! ONCE AND FOR ALL!

By all means, send more patrols to the nice, rich neighborhoods. I'm sure they'll be happy about more police presence. After all, their homes get extra protection and they can avoid the bad parts of town. Just don't blame the cops when the problem areas deteriorate even further.

flawed

[fluffernutter]

If it claims to use AI for something that cannot be programmed, then it is fundamentally flawed at least for the next 25 years or so.

Taboo topics

[Anonymous Coward]

Today it is taboo to discuss that few minority populations responsible for 90%+ of all urban crime. Without these models, police would be faced with wasted effort of policing no-crime white suburban neighborhoods instead of trying to stop black gangs from killing each other and selling drugs to black children.

Re:Taboo topics

[LynnwoodRooster]

Do you want to pay for their education?

Do you pay property taxes, or live in a building where the owner pays property taxes? Then you're paying for their education already.

Everyone knows about the "US spends more per capita on healthcare and we have crappy service!" concept, but how many know that we spend just about the most per student on education, and have terrible results?

Perhaps the issue isn't the amount of money spent in these areas, but the goals/intent of the Governments who are doing the spending (which is valid for both cases, as Government spending in the US - Federal and State - accounts for about 65% of all healthcare spending). They do not have success as their goal, but rather continuation of their own bureaucratic existence.

Re:Taboo topics

[LynnwoodRooster]

Wages are up ~4.4% over most of the last year.

Re:Taboo topics

[fluffernutter]

Supply and demand should have ALREADY raised the rates.

Re:Taboo topics

[fluffernutter]

I meant post-secondary education, whether it be a trade school, journeyman, college, whatever.

Re:Taboo topics

[fluffernutter]

Would you believe Forbes? In this climate, companies should have long raised wages. There should literally be no minimum wage jobs any longer in a plage with a "help wanted" sign in the window. Where should be NO H-1Bs except for skills that do no exist in America. Yet they still complain, they still continue to use H-1Bs.

Re:Taboo topics

[ceoyoyo]

Education is also the second most cost-effective method for reducing crime. You want to make sure you're providing enough of it, efficiently.

Re:Taboo topics

[fluffernutter]

Maybe what I'm trying to say is, these people commit crimes because no one wants them.

Re:Taboo topics

[fluffernutter]

I'm really sorry but just one more thing: I found it humorous that CNBC says "Trump will raise wages"; and you're right I think it's total bullshit.

Re:Taboo topics

[LynnwoodRooster]

It is a bit old, but this paper suggests that completing high school results in a significant drop in criminal activity - and that post-secondary education has little impact on criminal behavior (figure 1, page 32). Post-secondary education isn't a driver here - it's K-12, where we dump more than just about any other nation, and we still come well below the OECD average for graduation rates.

Re:Taboo topics

[LynnwoodRooster]

That's the problem; we're number 3 in the world in spending per student, and we're 17th out of 22 in the OECD in high school graduation rates.

We should pay more attention to Eisenhower's warning about the Government-Education complex, and how it's feeding off of the US taxpayer as much as the military-industrial complex. For example, the DOD budget in 2016 was $585 billion; Federal and State spending on K-12 education in 2016 was $620 billion. How many people would guess we spend more on education of K-12 than we do on the military?

When DOD cost overruns are found, there is much howling about cutting back to the DOD, increasing oversight, etc. When the educational system fails (as it does), there is handwringing and demand to spend even more. Somehow society has been conditioned to accept failure of the Government-Education complex and reward that failure with even more dollars and apologies.

Re:Taboo topics

[Mashiki]

You know what's funny? Is that US wages effectively stayed flat from ~2002ish until Trump was elected. Hell it was bad enough that the median income here in Canada was above the US median income. What changed though? Policy and action. Bush Jr, and Obama played the same game, ran the same types of policies, dealt with businesses the same way. More regulations, more restrictions, creating more uncertainty, granting more incentives for businesses to "offshore" or simply pick up and move out of the US. This isn't some type of genius rocket surgery by any means, Trudeau Jr., is doing the exact same BS that Obama did and guess what's happened up here. Within a few months of him taking office, wages stalled, unemployment started going for a shit, FT jobs started disappearing, more PT jobs in low-wage earner sectors started popping up. This is the same garbage the McGuinty and Wynne pulled in Ontario, same thing that Noltley is pulling in Alberta. Then we get a fundamental switch to policies and suddenly businesses are hiring FT workers, paying more and all the rest. Ontario accounted for ~65% of all job creation in Canada in the last quarter and over half of those piddly numbers were FT where as the rest of the country was PT with provinces like Alberta with 4% FT hires and 96% PT or seasonal-PT.

Re:Taboo topics

[RazorSharp]

The problem is that education is left in the hands of local entities. The most the federal government can do is offer grants for doing particular things. I know that when I was in school, much of the money went to computers that we didn't use. The one "computer class" I had was run by this lady who was completely unqualified to teach it.

Meanwhile, with the exception of the AP classes, class sizes remained very large. Standards were lowered because that increased the graduation rate, which made the administrators look good and kept those computer grants coming.

Until a Constitutional Amendment allows the federal government to really get involved in education, the quality of education one receives will be highly variable. If you look at the difference between our education system and those more successful ones it comes down to our education system isn't an education system, it's a bunch of little systems that operate in vastly different ways and the decisions are primarily controlled by those who shouldn't be qualified to make them.

Re:Taboo topics

[fluffernutter]

Not sure what you mean, the 'gig' economy is better than ever. My company has hired all contractors and there are so many looking for regular positions but none to go around.

Re:Taboo topics

[LynnwoodRooster]

So, your contention is that an educational system dominated by Congressional votes from California would work best for Utah and Nebraska? That a system that's geared towards large, urban areas is a sound approach for rural America?

Re:Taboo topics

[RazorSharp]

Well, in theory those California votes would be balanced out by Texas votes. I'm not saying that we need to get rid of FFA or that a federally run education system shouldn't ensure that the schools are tailored to their regions. Quite the opposite. I know you'll probably find this suggestion obscene, but I think that the legislation should provide the Department of Education broad leeway when structuring the school systems. This would allow for the rural and urban school to have radically different structures. Yes, I'm suggesting education be put in the hands of bureaucrats. But, as we saw with the Obama administration (or, at least as I would argue), bureaucracies can be highly effective when placed in the right hands. I would trust specialist bureaucrats over local politicians and PTA boards. The thing that's lacking from education in America is the flexibility to experiment. Locally run schools don't have the resources, political capital, or research capabilities necessary to radically experiment with the school structure, and hence most schools are structured no different than schools during the Victorian era where there's a teacher, chalk-board, and students at desks.

Re:Taboo topics

[LynnwoodRooster]

Those States with the fewest votes/Representatives at the Federal level, and with the largest percentage of rural districts, tend to have higher high school graduation rates. If anything, I would suggest that points to more local - rather than State or Federal - control of the schools.

Re:Taboo topics

[_Sharp'r_]

Nice theory, but back in reality the one school system ultimately under the governance of the Federal Government is that of Washington, D.C., where they spend $27K/student/year (twice the national average) in order to rank 2nd to last on math results and last on reading results.

Not sure our goal should be to replicate that "success" elsewhere.

Re:Taboo topics

[RazorSharp]

As someone who attended both a rural and an urban high school, I suspect those statistics don't say what you're implying. At the rural school, the average student did better. But I graduated ranked much higher than I would have at the urban high school with the same GPA. The highest performing students at the urban high school performed better, there were more AP classes provided, and I knew several people who went on to Ivy League or equivalent universities. In the rural high school, the graduation rate was much higher but the ceiling was lower. A lot of the extracurriculars that colleges care about didn't exist, while FFA and ROTC were really emphasized. That's not to say they didn't send kids to college, but they weren't sending a handful of kids to elite universities every year the the urban school was. Not that my anecdote is any more than an anecdote, but I think it at least calls your assertion into question.

But regardless of my personal experience, graduation rate is a rather poor metric. And even if it wasn't, both rural and urban areas have school districts controlled locally (either by the state, county, city, district or however the state set it up). Having more representatives at the federal level doesn't change the fact that, Constitutionally, the federal government is limited in the ways it can influence educational policy and funding. If those statistics say anything I would argue that they demonstrate the special challenges schools face in urban areas.

Re:Taboo topics

[LynnwoodRooster]

There is a direct link between crime and high school graduation; when someone graduates from high school, the odds of them being a criminal drop dramatically. If we're talking about lowering crime rate, the first thing we should do is ensure graduation from high school, since the correlation is so strong.

Wrong about data source

[fygment]

The data fed to the model is not just from police sent to neighbourhoods the model sends them to as suggested. The data is gathered from _all_ police dispatches. So the suggestion that the model is reinforcing its own biases is wrong.

Re:Wrong about data source

[malkavian]

Useful clarification! It'd be interesting to see if there's a rebuttal from the company that produces this, so that there's at least some form of rounded discussion, rather than the blunt hammer of "rouse the mob" that seems to go for journalism these days.

Re:Wrong about data source

[Mashiki]

That'll be interesting as well, because the company that did this based it off of previous long-hand methods of policing and predictive crime models. An example: In the old days, if you started seeing a rash of tire slashing or vandalism against vehicles you'd put more patrols through that area. This led to a decrease in crime in general, but could also lead to increased crime in other areas as police resources were shifted.

The entire premise of the model isn't new and the complaints about the "broken window" theory isn't unproven or controversial either. There's plenty of statistical data showing that the theory holds some or more then some value. The reality is, the more an area becomes rundown, the more likely that property values are depressed, and businesses will leave for other areas of the city as crime rises. This is effectively the opposite of gentrification of poor areas.

To give an example, in the late 1980's and through to the mid-1990's in London, Ontario you'd be insane to walk down Dundas St., even in daylight and that's the main street through a city with a population of ~200k at the time. Your chances of being mugged, stabbed, or having your store/vehicle burglarized was stupidly high. Back around '96 or so, after years of complaints from citizens, the fact that the entire downtown core was turning into a no-mans land with nothing but boarded up shops, they hired a police chief who took the exact opposite approach. This approach was very similar to the methods used in Manhattan during the "great crime purge" of the downtown. Today, in both cases you're pretty safe despite the rising crime rates due to other issues going on over the last decade.

If they really wanted to try policing methods on 'unproven and controversial' theories, then they should have gone with the "social strain theory" and put a fucking bullet it in when their ass hit the pavement. To put simply, for those that don't want to look it up. It boils down to "wealth bad, people become predisposed to crime because they don't have wealth." I'll point out that in the 1970's, this was the method of policing common in the USSR and GDR, and was such a obvious failure that despite the cold ware tensions and distrust of the west. The various communist policing services of the era ran to Interpol who in turn put them in touch with policing services in Canada and the US, that used broken window and the theories from the Chicago school(not perfect but it did do a lot to deal with crime foundational issues).

Re:Wrong about data source

[AmiMoJo]

Except that if more police are dispatched one some areas than others then naturally they will detect more crime in those areas. Most crime is unreported.

algorithms

[argStyopa]

I have mixed feelings about this.
First, the idea that algorithms alone can 'predict' something as subjective, human, and impulse-based as crime is ridiculous, and (I believe) born of a Utopian idea that taking people out of the equation can somehow remove bias, racism, and subjectivity from the process leaving some sort of idealistically mechanical, sterile system. For anyone who's worked in policing, crime prevention, or law enforcement fields, this should be a staggeringly stupid idea. Who's writing such algorithms but other people? On top of that, I expect there are now encrusted layers of ideology, in which results that don't conform to some utopian ideal of demographics are claimed to be 'racist' and formulaically 'corrected' to suit political goals, regardless of the facts of reality.

OTOH there is ABUNDANT work that shows that recidivism, particularly in the worst crimes, is concentrated in a surprisingly small number of individuals. I worked for a police dept where the longest serving officers maintained that 80%+ of the crimes were committed by a handful of families in the 50,000+ person city.

https://www.politifact.com/tex... lists some examples:

University of Pennsylvania criminologist Marvin Wolfgang tracked nearly 10,000 boys born in 1945 and living in Philadelphia from age 10 through 17; they ultimately gauged how often each boy came in contact with police for an offense. One upshot: 627 boys, 6 percent of the group, each accounted for five or more offenses, according to police reports. Those boys, Wolfgang wrote, were collectively identified as responsible for 52 percent of all the offenses recorded in the study and, he said, about two-thirds of all violent crimes believed to have been committed by the juveniles. In Patrick-speak, Wolfgang found that 6 percent of juvenile boys accounted for about half of alleged juvenile crimes.

The follow-up study, presented in progress in 1982, tracked more than 28,000 boys and girls born in 1958 who lived in Philadelphia from age 10 through 17. Among males, the study found, 61 percent of reported offenses were committed by 1,030 "chronic recidivists," comprising 7 percent of males in the study. That is, 7 percent of the boys accounted for 61 percent of the juvenile offenses.

In 2014, Swedish researchers drawing on records accounting for the experiences of 2.5 million people born in that country from 1958 to 1980 reported that from 1973 to 2004, some 1 percent of the population accounted for 63 percent of all violent crime convictions.

So it's clear that if we could identify this small percent and aggressively police them, we could make a sizable impact on crime.

Re:algorithms

[alvinrod]

I’m not particularly familiar with this system, but I’d doubt it’s working on anything other than a neighborhood basis. It isn’t predicting that John Die will commit a crime, rather than someone in John Doe’s neighborhood will. Maybe this system is being oversold, but it sounds like something an actuary would be able to tell you.

The elephant in the room is that it probably sends more police to minority neighborhoods. But that’s where the crime is at and it isn’t racist. The studies that have done regression analysis on crime show that it isn’t race, but socioeconomic and family group status that explain crime levels far better. So the police are going to poor neighborhoods with a lot of single mother families. That oredicts more crime regardless of the skin color of the people living there. Unfortunately everyone tunnels on it being a black or Latino neighborhood because they’re incapable of seeing a problem beyond a skin-depth level.

"controversial"

[cascadingstylesheet]

a predictive policing software that once cited the controversial, unproven "broken windows" policing theory as a part of its best practices.

The only thing "controversial" about broken window theory was that it worked.

The fact that it worked really bothered a certain class of people (people who wanted crime to be the fault of something other than criminals).

Re:"controversial"

[kenai_alpenglow]

I don't know if it was actually "proven", but evidence suggested (STRONGLY) that it worked. Corollary: tear down your crack houses, and the druggies will move on. Also, if someone is fixing those broken windows, there are people in the area. And those tend to drive out the criminals.

Re: You mean Profiling

[Anonymous Coward]

Yes, but profiling is racist.

This is created by a machine using aggregated information of reports of crime (which do not require police presence).

Alternative explanation: humans already know exactly where crime is most likely to happen, but acting as if they know is forbidden. This technology allows them to acheive their goal while having deniability about how they selected the target areas.

To speak plainly, black people commit a lot of crime.

Re: You know you're an academic

[nevermindme]

Within your race and community, for most people, it may much easier to spot the social tells of a bad physical encounter walking up to you. You are not walking up to me in colors to sell me amway.

*sigh*

[jythie]

So.. while not an academic, this is pretty close to my field of research. Looking at their model, I am not surprised they sold this product but deeply disappointed. This is the type of model that is REALLY easy to sell to people, both law enforcement and the military (our customer) are enamored with them for their near magic ability to 'predict' things. Only they don't, they tend to fail in unpredictable ways. They are not bad in multi-model systems where you take a dozen or so different systems built by different teams, run them in parallel, then have subject matter experts ponder the conflicting results. But actual police out of a single model? Madness... or hubris.. or stupidity... or simply being enamored with a slick sales pitch from 'one of your own' offering to solve problems in the way you want them solved.

Oddly enough, we actually DID do a LEO model years back, which was actually pretty effective, but it encouraged things like community outreach and police/citizen interaction which worked really well for officers on the ground but pissed off lawmakers and 'police unions', so it was largely dropped.

Which gets back to this story and one of the fundamental flaws in such attempts. The decision makers are not interested in solutions that make things better for high crime areas in the first place, the people in those areas are not part of their power block. They want solutions that 'sound right' to people who live elsewhere and confirm what they already believe. Which is exactly what models like this are good at producing. They are kinda like torture... useless for prediction or information gathering, but an excellent political tool for confirming the story your career depends on being 'true'.

Re:*sigh*

[apoc.famine]

Oddly enough, we actually DID do a LEO model years back, which was actually pretty effective, but it encouraged things like community outreach and police/citizen interaction which worked really well for officers on the ground but pissed off lawmakers and 'police unions', so it was largely dropped.

For several years I lived in walking distance between a police station and a neighborhood responsible for the vast majority of the police calls in the area. I could see the police station from my house, and the troubled neighborhood was a 10 minute walk behind me. I never, not once in five+ years, saw a cop on foot in my neighborhood.

For cops to be effective, they actually need to be part of the community. They can't just be a faceless, quasi-criminal organization which is not accountable to the citizens around them, and who those citizens don't know. Given their abuses and utter lack of engagement, I developed a healthy disdain for the police, and a fair bit of distrust for them. If that's where a white, middle-class attitude ends up, I can't imagine how poorly they were viewed by the folks who they actually came into contact with viewed them.

We don't need algorithms. We need police willing to be part of a community who are trusted to be fair and respected for being helpful, and not police viewed with fear and disrespect for being unaccountable murderous thugs and criminals who think they're better than everyone else. While Cracked isn't exactly an authoritative source, they had a very interesting interview with a brittish bobby at one point. What was really eye-opening is how much training they got to deal with people, and how "The one lesson that everyone who ever became a half-decent copper did take on board was that the most important piece of officer safety equipment we ever had was talk." It's a really different story when cops are asked to be good for a community and when they don't have quotas and metrics and all sorts of things they can arrest people for.

Re:*sigh*

[Mashiki]

Welcome to Canada, 1975. It took people moving up the chain of police forces to make the change back to a more traditional form of policing. It hadn't fully changed over to the inverse model here until about 1996 or so, even at that the RCMP still operates in the traditional model of top-down. The US like many western nations went heavy-in on the whole "roving police cars is a great idea, especially from great big centralized police stations!" The method you're talking about, with community outreach, letting officers/constables deal with the issues and whatnot stem directly from the warm bodies filling warm chairs at the top, and that's where the change has to happen. The big problem of course, is that by the time someone gets to the top they may no longer care becoming so fundamentally jaded by decades of what's been happening that they say fuck it.

Japan though, was one of the few countries out that didn't move to the "roving cars great idea" line of policing, but maintained a traditional kiosk type system. Effectively, police live and work in the area they patrol. They know the people, people know them, and it makes it easier to recognize both newcomers and people who normally aren't there. It requires more warm bodies, that means full time officers plus community police/recruits - but that also means more trust between the public and the police itself.

Re:*sigh*

[Mashiki]

To be frank, British policing has devolved so badly in the last 30 years that you can't even call it policing. You've got serious issues with the police and local government covering up crimes, with approval of higher-up members of the government. And arresting people for daring to question their actions, followed by rampant political correctness and far too much "well because they're not white..." reasoning in not laying charges. To use it as an example of anything, is to highlight what happens when a government looks at the social contract and goes "fuck you peasants." That in itself is what happens when countries are about to fail or become authoritarian.

You'll get a far more authoritative source by going off and doing a ride-along or three, or volunteering.

Re:*sigh*

[Mashiki]

As usual you have no idea what you're talking about. Thw worst stories you hear on the news are not anything like the average case.

If you think the 70's were notoriously corrupt, the only person who has no idea on what they're talking about is yourself. I'll give you bonus points for at least trying, let me know how much you're liking the fact that unlike in the 1970's, they now let violent criminals simply walk out the door.

Major fail by Mr. Academic...

[LynnwoodRooster]

Police typically respond to calls, not just cruise around and "spot" crime. The fact they're in a neighborhood cruising already does not increase the likelihood of a call coming in; it may shorten response time, but it does not cause the call in the first place. If anything, it would tend to depress illegal activity in that neighborhood...

Doesn't this just show

[rsilvergun]

that violent crime is less common then we like to think? Of course a small percentage is responsible for all violent crime. There isn't enough to go around.

Also, not sure about Sweden in the 80s but in America, even today, our prison system chews you up and spits out broken people. That's been equally well documented.

Finally pre-90s is a bad place to get crime statistics from. Lead in the air was pretty obviously creating unhinged people. Again, there's plenty of studies to back this up because it's the only thing that can explain the across the board drop in crime.

Re:Doesn't this just show

[Mashiki]

Why don't you take a look to Canada, which uses a similar method as EU countries in terms of dealing with recidivism. The reality is, the recidivism rate is still around 80% and that's after all the various programs ranging from training to psychological treatments have been applied. Got a lot of people in prison up here for child rape, violent assault, rape, and a lot of people not getting put into prison despite 200+ convictions for violent assaults because "they're natives(aka indigenous population)" aka "government bad, systemic oppression(society) made me into the way I am!" bullshit going on from the gladeu report.

The US has a serious problem with black crime(broken homes, generational, no-opportunities, open government handouts, ghettoization, substance abuse, etc), but nobody really wants to look at it and go "well what the fuck are we doing and throwing money at this shit for, instead of doing concrete things that nobody will like." Here in Canada, we have the exact same problem with natives, with exactly the same issues. You might have heard about the indigenous missing women's investigation and whatnot, if not simply put: Indigenous women were disappearing at a very high rate, never to be seen again. The previous report, much like the current investigation is a rehash of the same thing. Nearly 90% of the disappearances were directly attributed to issues within the community. In other words, they were killing their own women and there's probably a handful or two of indigenous serial killers that prey exactly on that group because so many go missing. Again, nobody wants to actually look at this in an actual way - because when someone does the first response from progressive media is to start screeching "that's racist."

Finally pre-90s is a bad place to get crime statistics from. Lead in the air was pretty obviously creating unhinged people. Again, there's plenty of studies to back this up because it's the only thing that can explain the across the board drop in crime.

That would be wrong, because during the 70's and early 80's with those "worst case" eras of crime, there was a shitload of other things going on. Everything from deflation to hyperinflation, to new 'social diseases' of the period such as heavy drug use and whatnot. The deflationary period where wages decreased by upto 25% in some cases over a 8 year span, followed by living costs exploding by 25% a year had a lot to do with the spike in all levels of crime. Lead is an easy way out, but the factors are far more reaching.

Re:Doesn't this just show

[Fnkmaster]

Demographic changes, abortion, technology (video games, cashless society, security systems), and economic changes. And maybe also some lead in the air.

That's not solving crime

[rsilvergun]

that's corralling it. Stick with me on this, it's long.

I saw this in my old town (I live in an apartment full of Indian H1-bs now and while they do take my jerbs they're about the least criminal people on earth if you don't count them hacking my cheap router from time to time).

I used to live in a cheap, working class neighborhood. Up the street were crack houses galore. Never once did they bother me. Only trouble I ever had was from a loser friend who's girlfriend robbed me.

I used to wonder why and then I found out. "Broken Window" policing is actually "Busted Heads" policing. My bud lived in one of those crack house apartments following a bad divorce (only thing he could afford/get after getting his credit destroyed). Got robbed, they caught the guy when his apartment manager went into the apartment and recognized his stuff there. They released the guy a few days later and he was still living next door to my bud when he finally got his credit fixed and moved out.

That was OK because they'd kept the crime inside. Every now and then one of them would venture out of their little hell hole and rob a liquor store or something.

The cops would come down like a ton of bricks. Everyone got arrested. And since they all had at least some pot half were probably gonna do a year or two in the clink. Especially the Dads, who would take the rap for the pot so the mom could at least stay out of prison. And during the raid you better believe heads got busted like crazy.

This kind of shit is used to force the lower caste to stay in their lane. Keep their head down. It's the nastiest form of oppression possible. It lets you and me ignore the problem of widespread poverty because when the poor make trouble there's a cop there ready to beat them the fuck down and a private prison system happy to lock 'em up for 3-5 years.

This is also why the drug war hasn't ended. Locking up randos for minor drug crimes is how you keep those folks on edge.

Now, you might be thinking, so what? They're criminals anyway. That's all well and good, but think about it. When Capitalism goes south what's suppose to fix it? The Answer is that Mr Factory Owner won't let the country go to shit because he lives in it. But Mr Factory owner and even his middle class servants can use tricks like this to control the populace why bother? What's to stop him from letting everything go to shit except where he lives? This is where oligarchy comes from.

Re: *sigh*... yes sigh, you are dumb

[jythie]

Yeah, those pesky domain experts and their actual knowledge of systems with proper tools of validation. So stupid.

A Minority of a Minority

[sycodon]

The fact is that the majority of the perps constitute a small minority of the minorities.

Re:A Minority of a Minority

[LynnwoodRooster]

Yep. But focusing on those perps is considered profiling and is verboten, so we have to pay just as much attention to the 87 year old grandmother to make things "fair".

Important crimes never reported to police

[swell]

Crime happens everywhere but it is not always reported. The worst, most critical crime in terms of its overall effect upon the public is never subject to a police dispatch. It happens at 1600 Pennsylvania Avenue and environs in Washington DC. It happens at Wall Street. And at Caracas, Mexico City, Dubai, Tel Aviv, Moscow, and even Geneva. It happens in marble buildings and fancy restaurants and hotels and involves people wearing Armani suits and fake sun tans. These crimes involve investment schemes, trade in drugs & guns, overthrowing governments, usurious loans to third world countries, manipulation of elections, monetary gifts to dictators and warlords, currency manipulations, deforestation for private profit and more. These criminals enrich each other at the expense of billions who suffer from the ravages of war, hunger, disease and deprivation.

The police are not called because these perpetrators are above the law.

ISSUE IS NOT REPORTING or "noticing crimes"

[denzacar]

Except the ISSUE IS NOT REPORTING or "noticing crimes" - IT'S PREDICTING CRIMES that's the issue.

Algorithm doesn't notice nor report crimes. It predicts where to send the police.
Resulting in a "garbage in = garbage out" predictive result based on reinforcement of outdated data.

E.g. If there was an arrest of a guy selling pot in front of a local Starbucks last month, and another guy arrested selling meth in a parking lot of a mall - algorithm now dictates through "near-repeat victimization" that both the Starbucks and the mall AND EVERYTHING AROUND THEM are likely locations of future crimes.
And should cops actually notice something in that area aroooouuund the location of a previous crime while being under pressure to fulfill their monthly quotas - it is seen as a validation of the predictive powers of the magical AI.
Rinse and repeat.

It's "Round up the usual suspects!" - only with locations and "supported" by math.

Pretty soon you have cops policing parking lots for broken tail lights and ID-checking everyone around a Starbucks, falling number of arrests for preventable crimes (such as selling drugs or opportunistic crimes) - with actual number of crimes on the increase city-wide.
Cause everyone is listening to the magical algorithm, designed to predict earthquake aftershocks.
Instead of having police patrolling even there where no crimes are being reported - e.g. cause the locals don't trust the police or are afraid of reprisals from the drug dealer next door.

Re:ISSUE IS NOT REPORTING or "noticing crimes"

[AHuxley]

Re "- with actual number of crimes on the increase city-wide."
That would be detected and added in.
Low crime, nice parts of a city would not suddenly become filled with crime.
Crime stays in parts of a city filled with criminals. Thats why the ability to track time works so well.

Re:ISSUE IS NOT REPORTING or "noticing crimes"

[denzacar]

Re "- with actual number of crimes on the increase city-wide."
That would be detected and added in.

No, it wouldn't.

Read the bit about people not reporting the dealer next door for the fear of reprisals.

Low crime, nice parts of a city would not suddenly become filled with crime.

Cause the "nice" people there wouldn't go to parts of the city where there are drug dealers but no cops, to buy drugs and take them home?
Or pay a dealer to do that and bring them drugs to their home?
Or cause they would report themselves for doing that?
Possession of drugs IS a crime.

Crime stays in parts of a city filled with criminals. Thats why the ability to track time works so well.

No.
And I'm not sure which sentence makes less sense.
The one which implies that criminals are paraplegics encased in concrete or the one which implies that we measure time with concrete-encased paraplegic criminals - instead of with clocks and calendars.

Not quite correct

[aepervius]

Some murder go unreported, or at least are reported as disappearance and not necessarily in the area they happened ! If somebody get ride of the body, and has 40% chance to get out of it some murder will necessarily never know where they happened) Furthermore many burglary are not reported due to various factor, fear from the person itself (maybe what was stolen was illegal, maybe the person was threatened, maybe the person is illegal, maybe the person don't trust the police, maybe what was stolen is low value etc....). Not all crime are reported or detected, not even murder (I won't even go into the resolve rate which is around 60% in average in the US). So yes even for burglary or murder, detection is a problem.

That went bad fast!

[Fringe]

I'm surprised that the normally-intelligent Slashdot readers didn't catch the overt bias and lies of this "study". Some of the more invective-laden responses clearly can't do English or math.

Did you catch this quote from the article? police shootings between 2011 and 2015 were 3.49 times more likely on average to target black individuals compared to white

Umm... gee... check the murder rates. Blacks kill blacks at a rate six times more than whites kill whites. Which means both they kill and they are killed. So the police could be considered as under-performing here. That stat was provided out-of-context, much like the rest of it. For example, the study... what did it say? Basically that areas with better data will have better results. Well, what did you expect?

This isn't a case of bad data making bad results. If nobody was committing a crime, there would be no feedback loop reinforcing the patrols. It's as if the study authors (and lamer Slashdotters) believe that catching criminals is wrong if it's in the wrong area! But no, what did the study consider the problem to be?

the over-patrolling of communities of color

How is this a problem, when those areas still are the higher-crime areas, and when patrols shouldn't bother you if you're not committing a crime?

You're missing the point

[rsilvergun]

they don't arrest them until they bother folks well above their social class. They're not enforcing the law, they're enforcing a twisted social order. That's why poor folk don't take kindly to the cops. They're not there to serve and protect. They're there to oppress and contain.

And no I'm not a "Fuck the Police" type. They're a product of the system. If nobody calls out the system's bullshit nothing changes.