Slashdot Mirror
New York City Moves To Create Accountability For Algorithms (propublica.org)
The algorithms that play increasingly central roles in our lives often emanate from Silicon Valley, but the effort to hold them accountable may have another epicenter: New York City. From a report: Last week, the New York City Council unanimously passed a bill to tackle algorithmic discrimination -- the first measure of its kind in the country. The algorithmic accountability bill, waiting to be signed into law by Mayor Bill de Blasio, establishes a task force that will study how city agencies use algorithms to make decisions that affect New Yorkers' lives, and whether any of the systems appear to discriminate against people based on age, race, religion, gender, sexual orientation or citizenship status. The task force's report will also explore how to make these decision-making processes understandable to the public. The bill's sponsor, Council Member James Vacca, said he was inspired by ProPublica's investigation into racially biased algorithms used to assess the criminal risk of defendants. "My ambition here is transparency, as well as accountability," Vacca said.
Please explain how an algorithm can be biased if you leave out ethnicity from the input data, but only after the fact discover that it results in fewer individuals of some group getting loans. It's not discriminating, it's just pointing out that two groups have very different input values as a very broad category. It probably also has different results between Asians, Jews, Hispanics, and most other groups. You're mistaking identifying different outcomes after the fact as a result of different initial factors with the usual human approach of lazily categorizing based on factors that aren't causal, but merely correlations.
You can even prove its not racist by finding a set of input data for individuals from two different demographic groups and seeing if it returns the same results for both. My guess is that it gives loans to black people who have good credit scores, a stable income, etc. and denies them to white people who have poor credit history and no steady income.
Algorithms are going to be far better than humans because they don't care about black, gay, atheist, etc. A human might well be intellectually lazy enough to group all blacks together as poor credit risks, but an algorithm isn't if you leave that irrelevant data out. In fact, using these algorithms would mean that if there is widespread discrimination against a group, that the company using the algorithm can actively pick out the people who will be able to repay loans which will generate additional profit. They've given themselves customers that other people are denying.
This doesn't look like being careful or taking preventative measures against misuse. Instead it reeks of not liking the results and not caring to address the underlying causes of those results. Giving loans to bad lending risks isn't going to magically make them responsible or more likely to pay back their loans. If black people, Methodists, or white people from WV happen to fall into that category more often than other groups, then you need to actually look at what is contributing to that result if you actually want to fix the problem.
Well, the issue I foresee in this effort is that while the algorithms will be perfectly fine, it's the policies created to make up for well functioning algorithms that will be the problem.
Because what policymakers will quickly find is that having equal algorithmic treatment or having equal standards for all does not lead to the outcomes they want, as people of different demographics, backgrounds, capabilities do not take up services or have success against different programs in the same way.
This is the problem with policy always -- a tendency to believe (at least in recent liberal democracy) that people are all drawn from the same starting set and have equal propensities for doing / being / acting / achieving / using certain things. And when policymakers find that to be the unavoidable truth, democratic pressure forces them to find ways around this truth and distort the outcomes.
No algorithm will get around that.