Slashdot Mirror
New York City Moves To Create Accountability For Algorithms (propublica.org)
The algorithms that play increasingly central roles in our lives often emanate from Silicon Valley, but the effort to hold them accountable may have another epicenter: New York City. From a report: Last week, the New York City Council unanimously passed a bill to tackle algorithmic discrimination -- the first measure of its kind in the country. The algorithmic accountability bill, waiting to be signed into law by Mayor Bill de Blasio, establishes a task force that will study how city agencies use algorithms to make decisions that affect New Yorkers' lives, and whether any of the systems appear to discriminate against people based on age, race, religion, gender, sexual orientation or citizenship status. The task force's report will also explore how to make these decision-making processes understandable to the public. The bill's sponsor, Council Member James Vacca, said he was inspired by ProPublica's investigation into racially biased algorithms used to assess the criminal risk of defendants. "My ambition here is transparency, as well as accountability," Vacca said.
A very good book that discusses the problems behind the blind implementation of algorithms is Weapons of Math Destruction by Cathy O’Neil.
I read an article about this kind of problem awhile back, only the algorithm being discussed was used by court systems to project the risk of a person becoming a repeat offender. A major problem with the system was that it was being used in ways that didn't match its intended use. But there were also real problems with the training data that was used. Historic racism for example distorts crime statistics for as long as they are viewed as relevant. Even today you have programs like 'Stop and Frisk' which perpetuate racist policing and all the resulting prosecutions from that continue to weigh the statistics down.
None of that should be surprising, and I'm not really against using algorithms for helping to make decisions. But those algorithms should not be black boxes, especially whenever they are used by government or institutions backed by government. And there should always be a route for an individual to obtain a breakdown of the algorithms analysis pertaining to them so that it can be contested when flawed.