Slashdot Mirror

← Back to Stories (view on slashdot.org)

Women Get Pull Requests Accepted More (Except When You Know They're Women) (peerj.com)

Posted by yaelk on from the women-in-the-workplace dept.

An anonymous reader writes: In the largest study of gender bias [in programming] to date, researchers found that women tend to have their pull requests accepted at a higher rate than men, across a variety of programming languages. This, despite the finding that their pull requests are larger and less likely to serve an immediate project need. At the same time, when the gender of the women is identifiable (as opposed to hidden), their pull requests are accepted less often than men's.

26 of 293 comments (clear)

  1. Just a thought... by ClickOnThis · · Score: 3, Insightful

    Maybe women ask for pull-requests more nicely?

    --
    If it weren't for deadlines, nothing would be late.
    1. Re:Just a thought... by ClickOnThis · · Score: 5, Informative

      You may want to go read the article. Unless you think being nicer works against you on a free software project. If that is the case then I may agree with you.

      Fair enough. For the TL/DR crowd, here are some of the possible explanations presented by the authors:

      - Reverse-discrimination against men? Rejected, per the observation that there is evidence of discrimination against women when gender is identified.

      - Women take fewer risks, and thus are more likely to provide solutions that are accepted? The authors cite a study that claims women are, on average, more risk-averse than men. However, this is inconsistent with the observation that women change more lines of code.

      - Women in open-source are more competent than men? This is the hypothesis that the authors support the most. They suggest it somes about due to survivorship bias and/or self-selection and/or higher implicit performance-standards in the female population of open-source coders.

      --
      If it weren't for deadlines, nothing would be late.
    2. Re:Just a thought... by Darinbob · · Score: 4, Interesting

      Not all useful changes are treated the same. Bug fixes get higher priority, doing what the boss thinks is important gets more priority, infrastructure changes which overall are an improvement but which causes a need for others to fix code or learn something new tend to get lower priority. Smaller means easier to quickly understand and thus more likely to be accepted quickly. Logically some of these things getting lower priority are actually very important but get overlooked as they're not directly related to the immediate bottom line and quarlerly profits (in the corporate world anyway, though some of this exists in a slightly different form in open source).

      And that's sort of what they implied. Pull requests from women tended to be larger or less likely to serve an immediate need. This is not to say that those are better or worse on merit, just treated differently.

      To stereotype perhaps, the women tend work on things that need to get done in the long run and avoid quick and dirty fixes, men tend to work on things to impress the boss and worry about cleaning it up later? I have seen some small trend this way in my experience, as the worst code bases to maintain that I've worked on tended to be developed in all male groups, and easier to understand and maintain code came from mixed developers. And in my experience at least, I've see more women caring about long term architectural issues and few who were engaged in the quick and dirty check in.

    3. Re:Just a thought... by wyHunter · · Score: 3, Insightful

      And usually those people aren't white men.

    4. Re: Just a thought... by loufoque · · Score: 4, Insightful

      I systematically google anyone who sends me a pull request. I assume most people do the same.
      Why wouldn't you be curious about that person that not only uses your software, but also took the time to fix a bug in it?

    5. Re:Just a thought... by Dr.+Evil · · Score: 4, Insightful

      ", per the observation that there is evidence of discrimination against women when gender is identified."

      Not sure how they come to this conclusion when they indicate that when the gender is identified, BOTH genders see a significant drop and men see a *greater* drop when they're known to the project. It's only when the women are unknown that their acceptance rate is lower... but even then, the acceptance rate of men and the acceptance rate of women's error bars overlap... it's entirely possible there's no difference between the genders when the contributor is unknown.

      In fact, the only place in their pull request acceptance rate error bars don't overlap on p15 is where identified male insiders are rejected at a greater rate than women.

      "We hypothesized that pull requests made by women are less likely to be accepted than those made by men."

      Seems like bad research... start with a hypothesis and highlight areas of your study which weakly support it, ignore areas which strongly refute it.

  2. We're not all career programmers. by Anonymous Coward · · Score: 3, Insightful

    What is a pull request? Is it a good or bad thing?

    1. Re:We're not all career programmers. by Verdatum · · Score: 4, Informative

      It's a term related to git, the tool a lot of us use to manage our source-code and revision history. A pull request is when you finish a task and you send your code changes up to the authorities of the project. When a pull request is approved, it means their code changes have been applied to the project.

    2. Re:We're not all career programmers. by bluefoxlucid · · Score: 4, Informative

      For distributed version control systems like git, mercurial, bazaar, bitkeeper, and darcs, there's no central repository. You can have an authoritative source, which is just like every other source aside from a fancy name tag. A pull request is a request to pull (and merge) a branch from another repository.

      --
      Support my political activism on Patreon.
    3. Re:We're not all career programmers. by tnk1 · · Score: 4, Informative

      A pull request is a definitely a "git-ism". It's a request to other coders to update their own local git codebase to incorporate the changes that the requester has made. So it is like a "request to commit" to some degree, but allows for decentralization.

      So, you can accept a pull request to your own personal branch/fork and it doesn't have to go on the main branch. This allows two (or more) coders to sync their branches with each other, without necessarily impacting the main branch. Then at some point, when there is full agreement among the collaborators about what they want to submit to main, the merged branch with all their work (or any one of the up-to-date branches) has a PR generated for it, and the request is made to update the main. (Or perhaps their branch just becomes a fork of the original code and now that branch is "main").

      Obviously, if the PR is accepted to the main, there could be rules about who can do it and/or under what circumstances. There may be a main branch committer, or there could just be rules to allow anyone to commit, as long as they aren't the author and that they have verified the changes meet the appropriate code review and testing requirements. There's no actual difference in the mechanical aspects of it; the main branch works just like any other branch aside from the designation of that branch as the "authoritative" code base for the builds and release candidates.

  3. Self-Selection? by Diss+Champ · · Score: 5, Insightful

    Is it possible that those women who don't feel it necessary to point out their gender in situations where gender doesn't matter tend to also be those more likely to communicate well?

    Is it possible that those women who make it a point to draw attention to their gender in situations where there is no reason to bring up gender at all, are also more likely to be less convincing regarding the usefulness of their work?

    1. Re:Self-Selection? by Pseudonymous+Powers · · Score: 5, Insightful

      Interesting point. Also worth asking is:

      Is it possible that those developers who don't feel it necessary to point out their favorite college sports team in situations where their favorite college sports team doesn't matter tend to also be those more likely to contribute worthwhile changes? Is it possible that those developers who make it a point to draw attention to their favorite college sports team in situations where there is no reason to bring up their favorite college sports team at all, are also more likely to be less convincing regarding the usefulness of their work?

    2. Re:Self-Selection? by Pseudonymous+Powers · · Score: 4, Funny

      Go Ferrets!

    3. Re:Self-Selection? by lgw · · Score: 4, Insightful

      possible that those developers who don't feel it necessary to point out their favorite college sports team in situations where their favorite college sports team doesn't matter tend to also be those more likely to contribute worthwhile changes?

      The double-negative makes it hard to parse, but I think I agree: "people who point out unimportant distractions about themselves have lower-quality submissions". Seems perfectly reasonable to me.
       

      --
      Socialism: a lie told by totalitarians and believed by fools.
    4. Re:Self-Selection? by ceoyoyo · · Score: 5, Interesting

      First impression: somebody needs to learn about statistics that have more than one predictor variable.

      Second impression: despite the lack of appropriate analysis, the differences in figure 5 are big enough to be reasonably clear. It looks like there is discrimination against anybody who has a gendered profile (maybe maintainers don't like pictures?). This discrimination might be slightly greater against outside women, and is fairly likely greater against inside men.

      Third impression: the paper and the Slashdot summary have a strong gender bias; they mention only the small and borderline significant anti-female bias while ignoring the more significant anti-male bias and also the much larger anti-(either) gender identifiable bias.

  4. Re:What about me? by Anonymous Coward · · Score: 5, Funny

    Have you tried Craigslist?

  5. RTFA ... by Obfuscant · · Score: 3, Funny
    Read TFA, at least through the "author contribution" section.

    Clearly, Clarissa didn't contribute anything, and Chris may or may not have contributed anything significant, it's hard to tell.

  6. What a crap summary by avandesande · · Score: 4, Insightful

    After reading the article it appears that women lead pull acceptance in every case except for one edge case, and not by very much(its like 64% vs 63%). Nothing interesting at all here.

    --
    love is just extroverted narcissism
    1. Re:What a crap summary by alexhs · · Score: 3, Insightful

      To be fair, Slashdot's summary is not worse than the paper's summary.

      There's a long list of issues with their methodology, and they make a fair assessment of these in the "Threats" part, which BTW should be discussed in the article, and not in the appendices.

      As a whole, this paper reeks "We wanted to show how / how much women were discriminated against in Open Source. Our findings showed the opposite, so we kept making up criteria until one would exhibit (barely) the bias we wanted to denounce."

      Of course when you're doing that, you're just begging to fall for this.

      Non-exhaustive list of other issues I noticed:
      - Weighing issues: for example, how many commits from outsiders vs insiders. Given that, overall, women get better acceptance, I can conclude than insiders commit more than outsiders (in their dataset)
      - Missing stats (for example, we get gendered stats on whether a pull request is linked to an issue, but no insider / outsider distinction)
      - Plain old lies in the summary ("when a woman’s gender is identifiable, they are rejected more often" vs "Women have lower acceptance rates as outsiders when they are identifiable as women.")
      - Failure to mention that the error bars are for the strict dataset. I suppose this is standard practice, but the dataset error bars are probably swamped by the non-representativity of the dataset in the first place, and the methodology shortcomings, which means that they're misleading (nobody cares about their dataset). They don't make any effort to evaluate these errors (obviously that would be the hard part), and leave us with some hand-waving like "we are somewhat confident that robots are not substantially influencing the results".
      - Graphs that start at 60% to exaggerate differences (without using broken axis)
      - Using "theory" for "hypothesis"

      --
      I have discovered a truly marvelous proof of killer sig, which this margin is too narrow to contain.
  7. Problem with this article by Verdatum · · Score: 3, Informative

    I honestly don't mind submissions about gender issues on /. But I do have a problem with posting articles that have not yet been peer reviewed. It is at least good of the link to make that perfectly clear.

  8. Re:oh ffs already by Rei · · Score: 3, Funny

    If you killing yourself would mean I'd see the acronym "SJW" less often on Slashdot, then by all means go right ahead.

    --
    We should start dealing in those black-market beagles.
  9. Baloney Charts by avandesande · · Score: 5, Interesting

    Charts that show a percentage range (ie. 60% to 80%) instead of the actual percentage (0% to 100%) to exaggerate differences between amounts on the chart.

    --
    love is just extroverted narcissism
  10. Stupid study is stupid by Anonymous Coward · · Score: 3, Interesting

    First off they trumpet the fact that they discovered that women's merge acceptances were higher than mens. It's only when they sliced the one hundred thousands of accounts for "gender confirmation" that they decided that bias existed because success rates went from 72% to 64% - The error deviation of that alone should cover the spread.

    Secondly the sample rating is awful - They compare TWO MILLION male checkins to ONE HUNDRED THOUSAND female checkins without any criteria for context, quality, need or style... just "quantity" and say that because the PERCENTAGE RATES FOR ACCEPTANCE are "higher" it must mean the women programmers are "Better" when comparing 2 sample sets with 20x the difference of checkins as they're all EQUAL.

    Sorry. That's BS.

    This is not science, this is propaganda statistics and poor statistics at that but I'm sure they made full use of their government funding to study gender issues in STEM fields.

  11. Re:oh ffs already by alvinrod · · Score: 3, Insightful

    It's a research study. If you have a problem with how the research was conducted or believe that the conclusions which have been drawn from the study are erroneous or the result of a particular methodological flaw feel free to point it out. Dismissing scientific results on the basis that you don't like them or people are using it for some political narrative isn't reasonable.

    Also, it doesn't look like anyone here is calling for diversity quotas or any other particular action. I'm sure some people will use this to point out why company X needs some program or some such stuff, but take umbrage with them or their policy, not the scientists who made an observation.

  12. Bimodal distributions by goombah99 · · Score: 3, Insightful

    The magnitude of the bias reported isn't alarmingly high so some of the things you suggest and others might be reasonable to consider as origins of the difference.

    However, the change of the acceptance rate histogram from uni-modal to bi-modal when the gender is known for a woman seems to be much stronger evidence of gender bias.

    The bottom axis of the histogram is rate of code rejections for an individual, and the left axis is the number of individuals with that rejection rate. When gender is not known both men and women have dominantly high acceptance rates tailing off towards low accpetance rates. However when gender is know a sharp second peak at the 90% rejection rate shows up on the women's histogram but not the men.

    Thus I think what this study shows is that for the most part women work on code in ways that produces code more likely to be accepted. The fact that it tends to be longer and not something on the bug list may make their submissions different (more substantial infrastructure not defect fixes might be one interpretation). So I'm not inclined to conclude much from that. But the bimodality seems to be evidence of a strong gender bias among a small number of open source projects.

    --
    Some drink at the fountain of knowledge. Others just gargle.
  13. Cherry picked results by kevingolding2001 · · Score: 4, Informative

    The interesting thing the study actually found was that pull-request acceptance rates dropped for BOTH males and females when the gender of the requester could be inferred from their username or avatar picture. In some categories that rate dropped more for males, and in others the rate dropped more for females.

    But they ignored the drop in rates for males and considered only the drop in rates for females when jumping to their conclusion of "gender bias".