New York City Moves To Create Accountability For Algorithms (propublica.org)

The algorithms that play increasingly central roles in our lives often emanate from Silicon Valley, but the effort to hold them accountable may have another epicenter: New York City. From a report: Last week, the New York City Council unanimously passed a bill to tackle algorithmic discrimination -- the first measure of its kind in the country. The algorithmic accountability bill, waiting to be signed into law by Mayor Bill de Blasio, establishes a task force that will study how city agencies use algorithms to make decisions that affect New Yorkers' lives, and whether any of the systems appear to discriminate against people based on age, race, religion, gender, sexual orientation or citizenship status. The task force's report will also explore how to make these decision-making processes understandable to the public. The bill's sponsor, Council Member James Vacca, said he was inspired by ProPublica's investigation into racially biased algorithms used to assess the criminal risk of defendants. "My ambition here is transparency, as well as accountability," Vacca said.

Comments

Re:Now hold Trump accountable for TREASON

[HornWumpus]

Republican!

These people are obviously just fakes, making Democrats look unhinged.

But it's believable because the Ds _have_ lost control of their loonies. Unless the Ds check their lunatic fringe, Trump is good for two terms.

More idiocy

[alvinrod]

Algorithms don't discriminate if you remove the kind of data (race, age, etc.) that would allow them to make categorizations or judgments based on that data. But if you examine the results after the fact and reapply those labels and find some difference in outcomes, its because there is some difference in input, not a category identifier. If you find your algorithm thinks African Americans are a worse lending risk, it's likely because they're categorically less well off financially than other demographic groups, not because its racist against black people.

This kind of idiotic approach is just ignoring the actual underlying problems or differences in favor of trying to slap a band-aid on top of it to assuage guilty feelings. Worse yet, it prevents confronting the actual issues head on and is doomed to failure.

Re:More idiocy

[CanHasDIY]

If you're dealing with medicine, noting ethnic differences is important. Doctors understand probabilities and knowing when certain probabilities are elevated can significantly alter diagnostics and treatment to the benefit of the patient.

And yet, I guarantee some non-doctors out there will claim it's racist to only test black people for sickle-cell anemia. This is why we can't have nice things - we allow the ignorant people to have an equal voice to the knowledgeable.

Re:More idiocy

[alvinrod]

Please explain how an algorithm can be biased if you leave out ethnicity from the input data, but only after the fact discover that it results in fewer individuals of some group getting loans. It's not discriminating, it's just pointing out that two groups have very different input values as a very broad category. It probably also has different results between Asians, Jews, Hispanics, and most other groups. You're mistaking identifying different outcomes after the fact as a result of different initial factors with the usual human approach of lazily categorizing based on factors that aren't causal, but merely correlations.

You can even prove its not racist by finding a set of input data for individuals from two different demographic groups and seeing if it returns the same results for both. My guess is that it gives loans to black people who have good credit scores, a stable income, etc. and denies them to white people who have poor credit history and no steady income.

Algorithms are going to be far better than humans because they don't care about black, gay, atheist, etc. A human might well be intellectually lazy enough to group all blacks together as poor credit risks, but an algorithm isn't if you leave that irrelevant data out. In fact, using these algorithms would mean that if there is widespread discrimination against a group, that the company using the algorithm can actively pick out the people who will be able to repay loans which will generate additional profit. They've given themselves customers that other people are denying.

This doesn't look like being careful or taking preventative measures against misuse. Instead it reeks of not liking the results and not caring to address the underlying causes of those results. Giving loans to bad lending risks isn't going to magically make them responsible or more likely to pay back their loans. If black people, Methodists, or white people from WV happen to fall into that category more often than other groups, then you need to actually look at what is contributing to that result if you actually want to fix the problem.

Re:More idiocy

[Whorhay]

I read an article about this kind of problem awhile back, only the algorithm being discussed was used by court systems to project the risk of a person becoming a repeat offender. A major problem with the system was that it was being used in ways that didn't match its intended use. But there were also real problems with the training data that was used. Historic racism for example distorts crime statistics for as long as they are viewed as relevant. Even today you have programs like 'Stop and Frisk' which perpetuate racist policing and all the resulting prosecutions from that continue to weigh the statistics down.

None of that should be surprising, and I'm not really against using algorithms for helping to make decisions. But those algorithms should not be black boxes, especially whenever they are used by government or institutions backed by government. And there should always be a route for an individual to obtain a breakdown of the algorithms analysis pertaining to them so that it can be contested when flawed.

Re:Mirror

[cayenne8]

Hmm.....so, if these councils reviewing these algorithms that are finding actual bonafide trends, that happen to break along racial, sexual, [insert special interest here]...and that don't happen to fit the politically correct meme of the day, that they will insist these be thrown out?

So, facts....if inconvenient....are not to be used or trusted?

Hmm...isn't that kinda deleting the purpose?

Re:Mirror

[CanHasDIY]

Right.

Because apparently, in 2017, math became racist.

"Weapons of Math Destruction

[Gramie2]

A very good book that discusses the problems behind the blind implementation of algorithms is Weapons of Math Destruction by Cathy O’Neil.

wrong solution

[supernova87a]

Well, the issue I foresee in this effort is that while the algorithms will be perfectly fine, it's the policies created to make up for well functioning algorithms that will be the problem.

Because what policymakers will quickly find is that having equal algorithmic treatment or having equal standards for all does not lead to the outcomes they want, as people of different demographics, backgrounds, capabilities do not take up services or have success against different programs in the same way.

This is the problem with policy always -- a tendency to believe (at least in recent liberal democracy) that people are all drawn from the same starting set and have equal propensities for doing / being / acting / achieving / using certain things. And when policymakers find that to be the unavoidable truth, democratic pressure forces them to find ways around this truth and distort the outcomes.

No algorithm will get around that.

Re:Now hold Trump accountable for TREASON

[phantomfive]

I don't care if Trump goes to jail, but as an American, I don't consider Russia our enemy. From a practical standpoint, we stand to gain more by working with Russia than working against them.