Slashdot Mirror

← Back to Stories (view on slashdot.org)

Women Get Pull Requests Accepted More (Except When You Know They're Women) (peerj.com)

Posted by yaelk on from the women-in-the-workplace dept.

An anonymous reader writes: In the largest study of gender bias [in programming] to date, researchers found that women tend to have their pull requests accepted at a higher rate than men, across a variety of programming languages. This, despite the finding that their pull requests are larger and less likely to serve an immediate project need. At the same time, when the gender of the women is identifiable (as opposed to hidden), their pull requests are accepted less often than men's.

17 of 293 comments (clear)

  1. Just a thought... by ClickOnThis · · Score: 3, Insightful

    Maybe women ask for pull-requests more nicely?

    --
    If it weren't for deadlines, nothing would be late.
    1. Re:Just a thought... by alvinrod · · Score: 2, Insightful

      Another explanation (at least according to github) is that white women are some of the biggest barriers to progress.

      Perhaps it is simply the result of men being good feminists and rejecting pull requests from women in order to promote greater diversity and inclusion in tech. I don't know if the authors of the study also factored in race to their data analysis though.

    2. Re:Just a thought... by wyHunter · · Score: 3, Insightful

      And usually those people aren't white men.

    3. Re: Just a thought... by loufoque · · Score: 4, Insightful

      I systematically google anyone who sends me a pull request. I assume most people do the same.
      Why wouldn't you be curious about that person that not only uses your software, but also took the time to fix a bug in it?

    4. Re:Just a thought... by Dr.+Evil · · Score: 4, Insightful

      ", per the observation that there is evidence of discrimination against women when gender is identified."

      Not sure how they come to this conclusion when they indicate that when the gender is identified, BOTH genders see a significant drop and men see a *greater* drop when they're known to the project. It's only when the women are unknown that their acceptance rate is lower... but even then, the acceptance rate of men and the acceptance rate of women's error bars overlap... it's entirely possible there's no difference between the genders when the contributor is unknown.

      In fact, the only place in their pull request acceptance rate error bars don't overlap on p15 is where identified male insiders are rejected at a greater rate than women.

      "We hypothesized that pull requests made by women are less likely to be accepted than those made by men."

      Seems like bad research... start with a hypothesis and highlight areas of your study which weakly support it, ignore areas which strongly refute it.

  2. Re:oh ffs already by JoshuaZ · · Score: 2, Insightful

    Gender issues are a real and serious problem. And you don't need to be a "SJW" to get that. Inefficiencies introduced by biases are bad because they make less good code get written or accepted. This harms *everyone*. And understanding exactly how much of a bias there is and where there is bias or isn't bias is important. If there's no problem in a given area, then we should know about that so we can focus resources elsewhere. We don't lose by getting more good data about the situation.

  3. We're not all career programmers. by Anonymous Coward · · Score: 3, Insightful

    What is a pull request? Is it a good or bad thing?

  4. Self-Selection? by Diss+Champ · · Score: 5, Insightful

    Is it possible that those women who don't feel it necessary to point out their gender in situations where gender doesn't matter tend to also be those more likely to communicate well?

    Is it possible that those women who make it a point to draw attention to their gender in situations where there is no reason to bring up gender at all, are also more likely to be less convincing regarding the usefulness of their work?

    1. Re:Self-Selection? by Pseudonymous+Powers · · Score: 5, Insightful

      Interesting point. Also worth asking is:

      Is it possible that those developers who don't feel it necessary to point out their favorite college sports team in situations where their favorite college sports team doesn't matter tend to also be those more likely to contribute worthwhile changes? Is it possible that those developers who make it a point to draw attention to their favorite college sports team in situations where there is no reason to bring up their favorite college sports team at all, are also more likely to be less convincing regarding the usefulness of their work?

    2. Re:Self-Selection? by lgw · · Score: 4, Insightful

      possible that those developers who don't feel it necessary to point out their favorite college sports team in situations where their favorite college sports team doesn't matter tend to also be those more likely to contribute worthwhile changes?

      The double-negative makes it hard to parse, but I think I agree: "people who point out unimportant distractions about themselves have lower-quality submissions". Seems perfectly reasonable to me.
       

      --
      Socialism: a lie told by totalitarians and believed by fools.
  5. What a crap summary by avandesande · · Score: 4, Insightful

    After reading the article it appears that women lead pull acceptance in every case except for one edge case, and not by very much(its like 64% vs 63%). Nothing interesting at all here.

    --
    love is just extroverted narcissism
    1. Re:What a crap summary by alexhs · · Score: 3, Insightful

      To be fair, Slashdot's summary is not worse than the paper's summary.

      There's a long list of issues with their methodology, and they make a fair assessment of these in the "Threats" part, which BTW should be discussed in the article, and not in the appendices.

      As a whole, this paper reeks "We wanted to show how / how much women were discriminated against in Open Source. Our findings showed the opposite, so we kept making up criteria until one would exhibit (barely) the bias we wanted to denounce."

      Of course when you're doing that, you're just begging to fall for this.

      Non-exhaustive list of other issues I noticed:
      - Weighing issues: for example, how many commits from outsiders vs insiders. Given that, overall, women get better acceptance, I can conclude than insiders commit more than outsiders (in their dataset)
      - Missing stats (for example, we get gendered stats on whether a pull request is linked to an issue, but no insider / outsider distinction)
      - Plain old lies in the summary ("when a woman’s gender is identifiable, they are rejected more often" vs "Women have lower acceptance rates as outsiders when they are identifiable as women.")
      - Failure to mention that the error bars are for the strict dataset. I suppose this is standard practice, but the dataset error bars are probably swamped by the non-representativity of the dataset in the first place, and the methodology shortcomings, which means that they're misleading (nobody cares about their dataset). They don't make any effort to evaluate these errors (obviously that would be the hard part), and leave us with some hand-waving like "we are somewhat confident that robots are not substantially influencing the results".
      - Graphs that start at 60% to exaggerate differences (without using broken axis)
      - Using "theory" for "hypothesis"

      --
      I have discovered a truly marvelous proof of killer sig, which this margin is too narrow to contain.
  6. Re:Tugging by bluefoxlucid · · Score: 2, Insightful

    It means their code is less-important and so is not scrutinized as hard.

    --
    Support my political activism on Patreon.
  7. Re:oh ffs already by alvinrod · · Score: 3, Insightful

    It's a research study. If you have a problem with how the research was conducted or believe that the conclusions which have been drawn from the study are erroneous or the result of a particular methodological flaw feel free to point it out. Dismissing scientific results on the basis that you don't like them or people are using it for some political narrative isn't reasonable.

    Also, it doesn't look like anyone here is calling for diversity quotas or any other particular action. I'm sure some people will use this to point out why company X needs some program or some such stuff, but take umbrage with them or their policy, not the scientists who made an observation.

  8. Re:oh ffs already by Shadow+of+Eternity · · Score: 2, Insightful

    You're absolutely right. The fact 50% of domestic violence victims have 0% of federal funding and shelters, and 50% of rape victims aren't even legally recognized, is a real and serious problem.

    Manufactured "discrimination" about pull requests is neither real nor serious.

    --
    A bullet may have your name on it but splash damage is addressed "To whom it may concern."
  9. Bimodal distributions by goombah99 · · Score: 3, Insightful

    The magnitude of the bias reported isn't alarmingly high so some of the things you suggest and others might be reasonable to consider as origins of the difference.

    However, the change of the acceptance rate histogram from uni-modal to bi-modal when the gender is known for a woman seems to be much stronger evidence of gender bias.

    The bottom axis of the histogram is rate of code rejections for an individual, and the left axis is the number of individuals with that rejection rate. When gender is not known both men and women have dominantly high acceptance rates tailing off towards low accpetance rates. However when gender is know a sharp second peak at the 90% rejection rate shows up on the women's histogram but not the men.

    Thus I think what this study shows is that for the most part women work on code in ways that produces code more likely to be accepted. The fact that it tends to be longer and not something on the bug list may make their submissions different (more substantial infrastructure not defect fixes might be one interpretation). So I'm not inclined to conclude much from that. But the bimodality seems to be evidence of a strong gender bias among a small number of open source projects.

    --
    Some drink at the fountain of knowledge. Others just gargle.
  10. Re:oh ffs already by Mars+Saxman · · Score: 2, Insightful

    Yeah, at this point I pretty much just immediately flip the bozo bit on anyone who uses the term "SJW" non-ironically. It conveys no useful information except that the person using it is... um... possibly a troglodyte.