Slashdot Mirror
New York City Moves To Create Accountability For Algorithms (propublica.org)
The algorithms that play increasingly central roles in our lives often emanate from Silicon Valley, but the effort to hold them accountable may have another epicenter: New York City. From a report: Last week, the New York City Council unanimously passed a bill to tackle algorithmic discrimination -- the first measure of its kind in the country. The algorithmic accountability bill, waiting to be signed into law by Mayor Bill de Blasio, establishes a task force that will study how city agencies use algorithms to make decisions that affect New Yorkers' lives, and whether any of the systems appear to discriminate against people based on age, race, religion, gender, sexual orientation or citizenship status. The task force's report will also explore how to make these decision-making processes understandable to the public. The bill's sponsor, Council Member James Vacca, said he was inspired by ProPublica's investigation into racially biased algorithms used to assess the criminal risk of defendants. "My ambition here is transparency, as well as accountability," Vacca said.
Algorithms can discriminate even without the raw classifications. We saw a case on Slashdot earlier where a computer program took in data about people being sentenced and came up with a likelihood of re-offending in two years. When it was studied, and compared to actual two-year data about re-offending, it turned out to typically underestimate the chance for whites and overestimate the chance for blacks. That's clearly discriminatory. It gives significantly different results based on category.
If your algorithm consistently overestimates the lending risk for blacks and underestimates it for whites, as shown by empirical data, it's racist. It doesn't matter what the inputs are.
Algorithms don't necessarily reflect the real world. If they did, we'd have solved physics ages ago.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
What if, just maybe, beyond skin color, there are genetic differences between the races in how people value life, truth, and their propensity to violence. This is where people want to bury their heads in the sand. It's time we get honest and accept these truths.
Oh, bullshit. There's nothing "head burying" about wanting to treat people fairly. Even assuming the racial differences you posit exist (assuming race actually exists as a coherent and well-defined thing, which is debatable [1]), the racial differences are utterly swamped by individual differences, so it makes no sense whatsoever to make assumptions about individuals based on racial characteristics. Supposing, to take one example, African Americans score lower on IQ tests because they're not as smart, on average, as white Americans, rather than because the tests are culturally-biased. Does that mean Thomas Sowell is dumb?
The same applies across the board. Outside of the specific characteristics that we use to categorize people into races, all of the various potential statistical differences between groups are utterly dominated by individual differences.
And even if that weren't true. So what? Wouldn't you rather live in a world where every individual has an equal opportunity regardless of the elements of themselves that they can't change or control? For that matter, wouldn't you rather live in a world where society even attempts to make reasonable accommodation for individual differences, and even arguable deficiencies?
I sure as hell would.
So this is absolutely the right thing to do. Algorithmic decisionmaking is a fantastically useful and powerful tool that we're applying to improve human lives. Scrutinizing those algorithmic decisions to search for evidence of bias and then figuring out how to offset, mitigate or manage that bias is the right thing to do -- and that's true even if the bias has some legitimate statistical basis. And it's especially true if it does not, but we'll never know if we don't look.
[1] One of the most interesting studies of recent years about race came to the rather surprising conclusion that going to prison makes you black. A longitudinal study of racial self-identification and social identification found that many people who self-identified as non-black before being sent to prison self-identified as black after going to prison. Further, those self-identifications correlated strongly with third-party identifications by census workers, social workers and others who were asked to racially categorize people. People who are consistently non-black before imprisonment come out of prison as black. This is powerful evidence that a significant element of what we call "race" is a purely social construct.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.