Slashdot Mirror


New York City Moves To Create Accountability For Algorithms (propublica.org)

The algorithms that play increasingly central roles in our lives often emanate from Silicon Valley, but the effort to hold them accountable may have another epicenter: New York City. From a report: Last week, the New York City Council unanimously passed a bill to tackle algorithmic discrimination -- the first measure of its kind in the country. The algorithmic accountability bill, waiting to be signed into law by Mayor Bill de Blasio, establishes a task force that will study how city agencies use algorithms to make decisions that affect New Yorkers' lives, and whether any of the systems appear to discriminate against people based on age, race, religion, gender, sexual orientation or citizenship status. The task force's report will also explore how to make these decision-making processes understandable to the public. The bill's sponsor, Council Member James Vacca, said he was inspired by ProPublica's investigation into racially biased algorithms used to assess the criminal risk of defendants. "My ambition here is transparency, as well as accountability," Vacca said.

21 of 183 comments (clear)

  1. Re:Now hold Trump accountable for TREASON by HornWumpus · · Score: 3, Insightful

    Republican!

    These people are obviously just fakes, making Democrats look unhinged.

    But it's believable because the Ds _have_ lost control of their loonies. Unless the Ds check their lunatic fringe, Trump is good for two terms.

    --
    John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
  2. At least you can examine an algorithm by Rick+Schumann · · Score: 2

    More and more so-called 'AIs' are being used in place of algorithms (due mainly to magical thinking) but even the designers of these AIs can't tell you what they're really doing under the hood. That's where we're going to get in trouble with regards to 'accountability'.

  3. More idiocy by alvinrod · · Score: 4, Insightful

    Algorithms don't discriminate if you remove the kind of data (race, age, etc.) that would allow them to make categorizations or judgments based on that data. But if you examine the results after the fact and reapply those labels and find some difference in outcomes, its because there is some difference in input, not a category identifier. If you find your algorithm thinks African Americans are a worse lending risk, it's likely because they're categorically less well off financially than other demographic groups, not because its racist against black people.

    This kind of idiotic approach is just ignoring the actual underlying problems or differences in favor of trying to slap a band-aid on top of it to assuage guilty feelings. Worse yet, it prevents confronting the actual issues head on and is doomed to failure.

    1. Re:More idiocy by Anonymous Coward · · Score: 2, Insightful

      The more we learn about science, the more we are going to want to bury our head in the sand and ignore it.

      Yes, in medicine there are statistical differences between the races.

      What if, just maybe, beyond skin color, there are genetic differences between the races in how people value life, truth, and their propensity to violence. This is where people want to bury their heads in the sand. It's time we get honest and accept these truths.

    2. Re:More idiocy by CanHasDIY · · Score: 3, Insightful

      If you're dealing with medicine, noting ethnic differences is important. Doctors understand probabilities and knowing when certain probabilities are elevated can significantly alter diagnostics and treatment to the benefit of the patient.

      And yet, I guarantee some non-doctors out there will claim it's racist to only test black people for sickle-cell anemia. This is why we can't have nice things - we allow the ignorant people to have an equal voice to the knowledgeable.

      --
      An enigma, wrapped in a riddle, shrouded in bacon and cheese
    3. Re:More idiocy by alvinrod · · Score: 4, Interesting

      Please explain how an algorithm can be biased if you leave out ethnicity from the input data, but only after the fact discover that it results in fewer individuals of some group getting loans. It's not discriminating, it's just pointing out that two groups have very different input values as a very broad category. It probably also has different results between Asians, Jews, Hispanics, and most other groups. You're mistaking identifying different outcomes after the fact as a result of different initial factors with the usual human approach of lazily categorizing based on factors that aren't causal, but merely correlations.

      You can even prove its not racist by finding a set of input data for individuals from two different demographic groups and seeing if it returns the same results for both. My guess is that it gives loans to black people who have good credit scores, a stable income, etc. and denies them to white people who have poor credit history and no steady income.

      Algorithms are going to be far better than humans because they don't care about black, gay, atheist, etc. A human might well be intellectually lazy enough to group all blacks together as poor credit risks, but an algorithm isn't if you leave that irrelevant data out. In fact, using these algorithms would mean that if there is widespread discrimination against a group, that the company using the algorithm can actively pick out the people who will be able to repay loans which will generate additional profit. They've given themselves customers that other people are denying.

      This doesn't look like being careful or taking preventative measures against misuse. Instead it reeks of not liking the results and not caring to address the underlying causes of those results. Giving loans to bad lending risks isn't going to magically make them responsible or more likely to pay back their loans. If black people, Methodists, or white people from WV happen to fall into that category more often than other groups, then you need to actually look at what is contributing to that result if you actually want to fix the problem.

    4. Re:More idiocy by Mal-2 · · Score: 2

      It is racist to only test black people for sickle cell. The condition is common to areas outside Africa where malaria is still prevalent, you know.

      --
      How is the Riemann zeta function like Trump rallies? Both have an endless number of trivial zeros.
    5. Re:More idiocy by swb · · Score: 2

      The credit scoring industry is always eager to find one more factor they can include in calculating credit risk and they seem fond of high-correlation variables unrelated to actual loan performance, like driving record. I'm mostly convinced this is just to find a way to charge a premium to good credit risks.

      But there is only so much money good credit risks will borrow (which is partly why they're good credit risks, it's a kind of self-selective behavior) and lenders would like to loan more money in order to make more money. So they start looking for alternative credit scoring variables they can correlate to good loan risks, independent of past or no credit history.

      So you wind up with variables that turn out to be reasonable predictors of credit performance that are race-blind but wind up to coincidentally highly correlated with race, like zipcode. Happen to live in a majority black neighborhood? Based on broad economic measures, blacks probably are worse credit risks just because they're overall poorer, so just sharing the neighborhood makes you disadvantaged by a "blind" variable like zipcode, even if you're not black. If you are, it is indistinguishable from racial discrimination.

      Don't get me wrong, I don't disagree with your larger point. The problem really seems to be that "innovators" wind up using variables that seem neutral but are actually heavily biased. They should probably check for high levels of racial correlation when developing their algorithms and toss those that wind up correlating on race, or at least do so if it means preventing access to something.

    6. Re:More idiocy by GameboyRMH · · Score: 2

      By correlating other information, it's possible for a piece of software to be racist without using race as an input. You should give this a read:

      https://www.propublica.org/art...

      --
      "When information is power, privacy is freedom" - Jah-Wren Ryel
    7. Re:More idiocy by Khashishi · · Score: 2

      Well, in US labor law there's something called disparate impact. There is a grey area here and the ultimate answer will come from a social compromise, not from philosophy.

    8. Re:More idiocy by Whorhay · · Score: 5, Informative

      I read an article about this kind of problem awhile back, only the algorithm being discussed was used by court systems to project the risk of a person becoming a repeat offender. A major problem with the system was that it was being used in ways that didn't match its intended use. But there were also real problems with the training data that was used. Historic racism for example distorts crime statistics for as long as they are viewed as relevant. Even today you have programs like 'Stop and Frisk' which perpetuate racist policing and all the resulting prosecutions from that continue to weigh the statistics down.

      None of that should be surprising, and I'm not really against using algorithms for helping to make decisions. But those algorithms should not be black boxes, especially whenever they are used by government or institutions backed by government. And there should always be a route for an individual to obtain a breakdown of the algorithms analysis pertaining to them so that it can be contested when flawed.

  4. Re:Mirror by cayenne8 · · Score: 3, Insightful
    Hmm.....so, if these councils reviewing these algorithms that are finding actual bonafide trends, that happen to break along racial, sexual, [insert special interest here]...and that don't happen to fit the politically correct meme of the day, that they will insist these be thrown out?

    So, facts....if inconvenient....are not to be used or trusted?

    Hmm...isn't that kinda deleting the purpose?

    --
    Light travels faster than sound. This is why some people appear bright until you hear them speak.........
  5. Re:Mirror by CanHasDIY · · Score: 4, Insightful

    Right.

    Because apparently, in 2017, math became racist.

    --
    An enigma, wrapped in a riddle, shrouded in bacon and cheese
  6. "Weapons of Math Destruction by Gramie2 · · Score: 3, Informative

    A very good book that discusses the problems behind the blind implementation of algorithms is Weapons of Math Destruction by Cathy O’Neil.

  7. wrong solution by supernova87a · · Score: 3, Interesting

    Well, the issue I foresee in this effort is that while the algorithms will be perfectly fine, it's the policies created to make up for well functioning algorithms that will be the problem.

    Because what policymakers will quickly find is that having equal algorithmic treatment or having equal standards for all does not lead to the outcomes they want, as people of different demographics, backgrounds, capabilities do not take up services or have success against different programs in the same way.

    This is the problem with policy always -- a tendency to believe (at least in recent liberal democracy) that people are all drawn from the same starting set and have equal propensities for doing / being / acting / achieving / using certain things. And when policymakers find that to be the unavoidable truth, democratic pressure forces them to find ways around this truth and distort the outcomes.

    No algorithm will get around that.

  8. Re: Mirror by MBGMorden · · Score: 2

    That shouldn't really matter. If they are looking for educated people and more blue people are educated than green then the organization shouldn't have to worry about hiring less qualified people based on political correctness.

    Almost comically, these types of things also come from absolute hypocrites.

    If I say "Green people are less educated." I'm attacked for propagating a stereotype, yet the same people levying those attacks will say "You can't hire based on education because green people can't compete.".

    The stereotypes are both shunned and joyously embraced depending on whether one feels it'll help or hurt in the current scenario.

    --
    "People who think they know everything are very annoying to those of us who do."-Mark Twain
  9. Re:Now hold Trump accountable for TREASON by phantomfive · · Score: 3, Insightful

    I don't care if Trump goes to jail, but as an American, I don't consider Russia our enemy. From a practical standpoint, we stand to gain more by working with Russia than working against them.

    --
    "First they came for the slanderers and i said nothing."
  10. Re: Mirror by c6gunner · · Score: 2

    It may be true that people of whatever race are statistically more likely to be involved in crime, but it's not OK to deny an individual a loan on that basis, for instance.

    Where does that stop though? Statistically men die earlier than women; is it wrong to charge men more for life insurance? Statistically women cost the medical system more than men; is it wrong to charge women more for health insurance? Statistically men are more likely to be involved in a traffic accident; is it wrong to charge men more for car insurance?

  11. Re:Democrats are treasonous and love bump stocks by cayenne8 · · Score: 2

    When you're a homicidal maniac spraying bullets into a crowd, accuracy is secondary to rate of fire. Come on dude, this is mass shooting 101.

    Oh, I agree.

    Thing is...a bump fire stock, by nature of how it works....isn't really that great or reliable if you are trying to move around with your gun.

    If you are set up in a sniper area like he was, with multiple weapons fitted with them, to allow cooling and not having to reload as often and being somewhat able to stand stationary while using them, then they are dangerous as we saw.

    However, you really can't be moving, walking or running trying to use one in a crowd....they just don't work that well, again, by nature of how they work.

    So, aside from set ups like the LV shooter used, they generally aren't that effective for general use.....converting to FULL AUTO would be the best way to go, and of course, that is already ILLEGAL.

    And a bump stock doesn't increate rate of fire THAT much. There was a demonstration not long back, that showed:

    A regular semi-auto AR15 could be shot about 5 rounds per second.

    A bump stock AR15 could fire about 7.5 rounds per second.

    A Full Auto fired about 15 rounds per second.

    So....combined with their problems, that's why you've not seen much prior crime committed with a bump stock...and likely banning them won't prevent much either. It will primarily ONLY keep law abiding citizens from having a bit of fun at the gun range.

    --
    Light travels faster than sound. This is why some people appear bright until you hear them speak.........
  12. Re:Mirror by cayenne8 · · Score: 2

    That's kind of the problem with these algorithms: they are simply applying statistical facts without bothering about causation.

    But, if you're going for purely predictive results....what part does "causation" play in this at all?

    --
    Light travels faster than sound. This is why some people appear bright until you hear them speak.........
  13. Re: Mirror by JaredOfEuropa · · Score: 2

    Not an easy question, and I'm hardly an expert on legal or ethical matters. But it seems to me that it's unfair to discriminate on traits where there is only an indirect correlation with undesirable outcomes. If men die earlier than women because of physiological traits, then perhaps it's ok to charge them more for life insurance (though insurers and goverments might not do or allow that for other reasons). But what if black people die earlier? Statistically speaking that's probably the case, but there is no direct correlation between being black and dying earlier; a white guy in the same situation probably faces the same odds. Certain groups are generaly poorer and more likely to eat bad foods as a result, more likely to be in gangs, etc... but that has little to do with race and more with their average economic position. So charge them more for life insurance if they are actually in a gang, or eat crappy food, in other words for the stuff that has a direct impact on their life expectancy, but not for skin color.

    --
    If construction was anything like programming, an incorrectly fitted lock would bring down the entire building...