New York City Moves To Create Accountability For Algorithms (propublica.org)

The algorithms that play increasingly central roles in our lives often emanate from Silicon Valley, but the effort to hold them accountable may have another epicenter: New York City. From a report: Last week, the New York City Council unanimously passed a bill to tackle algorithmic discrimination -- the first measure of its kind in the country. The algorithmic accountability bill, waiting to be signed into law by Mayor Bill de Blasio, establishes a task force that will study how city agencies use algorithms to make decisions that affect New Yorkers' lives, and whether any of the systems appear to discriminate against people based on age, race, religion, gender, sexual orientation or citizenship status. The task force's report will also explore how to make these decision-making processes understandable to the public. The bill's sponsor, Council Member James Vacca, said he was inspired by ProPublica's investigation into racially biased algorithms used to assess the criminal risk of defendants. "My ambition here is transparency, as well as accountability," Vacca said.

Comments

More idiocy

[alvinrod]

Algorithms don't discriminate if you remove the kind of data (race, age, etc.) that would allow them to make categorizations or judgments based on that data. But if you examine the results after the fact and reapply those labels and find some difference in outcomes, its because there is some difference in input, not a category identifier. If you find your algorithm thinks African Americans are a worse lending risk, it's likely because they're categorically less well off financially than other demographic groups, not because its racist against black people.

This kind of idiotic approach is just ignoring the actual underlying problems or differences in favor of trying to slap a band-aid on top of it to assuage guilty feelings. Worse yet, it prevents confronting the actual issues head on and is doomed to failure.

Re:More idiocy

[alvinrod]

Please explain how an algorithm can be biased if you leave out ethnicity from the input data, but only after the fact discover that it results in fewer individuals of some group getting loans. It's not discriminating, it's just pointing out that two groups have very different input values as a very broad category. It probably also has different results between Asians, Jews, Hispanics, and most other groups. You're mistaking identifying different outcomes after the fact as a result of different initial factors with the usual human approach of lazily categorizing based on factors that aren't causal, but merely correlations.

You can even prove its not racist by finding a set of input data for individuals from two different demographic groups and seeing if it returns the same results for both. My guess is that it gives loans to black people who have good credit scores, a stable income, etc. and denies them to white people who have poor credit history and no steady income.

Algorithms are going to be far better than humans because they don't care about black, gay, atheist, etc. A human might well be intellectually lazy enough to group all blacks together as poor credit risks, but an algorithm isn't if you leave that irrelevant data out. In fact, using these algorithms would mean that if there is widespread discrimination against a group, that the company using the algorithm can actively pick out the people who will be able to repay loans which will generate additional profit. They've given themselves customers that other people are denying.

This doesn't look like being careful or taking preventative measures against misuse. Instead it reeks of not liking the results and not caring to address the underlying causes of those results. Giving loans to bad lending risks isn't going to magically make them responsible or more likely to pay back their loans. If black people, Methodists, or white people from WV happen to fall into that category more often than other groups, then you need to actually look at what is contributing to that result if you actually want to fix the problem.

Re:More idiocy

[Whorhay]

I read an article about this kind of problem awhile back, only the algorithm being discussed was used by court systems to project the risk of a person becoming a repeat offender. A major problem with the system was that it was being used in ways that didn't match its intended use. But there were also real problems with the training data that was used. Historic racism for example distorts crime statistics for as long as they are viewed as relevant. Even today you have programs like 'Stop and Frisk' which perpetuate racist policing and all the resulting prosecutions from that continue to weigh the statistics down.

None of that should be surprising, and I'm not really against using algorithms for helping to make decisions. But those algorithms should not be black boxes, especially whenever they are used by government or institutions backed by government. And there should always be a route for an individual to obtain a breakdown of the algorithms analysis pertaining to them so that it can be contested when flawed.

Re:Mirror

[CanHasDIY]

Right.

Because apparently, in 2017, math became racist.