Slashdot Mirror
Women Get Pull Requests Accepted More (Except When You Know They're Women) (peerj.com)
An anonymous reader writes: In the largest study of gender bias [in programming] to date, researchers found that women tend to have their pull requests accepted at a higher rate than men, across a variety of programming languages. This, despite the finding that their pull requests are larger and less likely to serve an immediate project need. At the same time, when the gender of the women is identifiable (as opposed to hidden), their pull requests are accepted less often than men's.
Maybe women ask for pull-requests more nicely?
If it weren't for deadlines, nothing would be late.
Can we stop it with yet another SJW troll story. Seriously. I get it, I know I'm supposed to kill myself since I have a penis.
I am a cisgender hermaphrodite who identifies as male. Nobody pulls me!
What is a pull request? Is it a good or bad thing?
Not even if I ask nicely.
Please pull my finger...
Is it possible that those women who don't feel it necessary to point out their gender in situations where gender doesn't matter tend to also be those more likely to communicate well?
Is it possible that those women who make it a point to draw attention to their gender in situations where there is no reason to bring up gender at all, are also more likely to be less convincing regarding the usefulness of their work?
"pull requests" heh.
Posted intentionally to lampoon typical responses.
I am not surprised that requests are not followed up on when a female calls for them, nor am I surprised that their responses are more often responded to when the gender is hidden/neutral. What I am surprised is that female pull requests are "larger and less likely to serve an immediate project need". Does this mean that female developers are concentrating on "big picture features" more often ?
If only we could fall into a woman's arms without falling into her hands
I read the summary and followed the link, but I still have no fucking idea what constitutes a "pull request".
Clearly, Clarissa didn't contribute anything, and Chris may or may not have contributed anything significant, it's hard to tell.
After reading the article it appears that women lead pull acceptance in every case except for one edge case, and not by very much(its like 64% vs 63%). Nothing interesting at all here.
love is just extroverted narcissism
I honestly don't mind submissions about gender issues on /. But I do have a problem with posting articles that have not yet been peer reviewed. It is at least good of the link to make that perfectly clear.
Charts that show a percentage range (ie. 60% to 80%) instead of the actual percentage (0% to 100%) to exaggerate differences between amounts on the chart.
love is just extroverted narcissism
Our results suggest that although women on GitHub may be more competent overall, bias against them exists nonetheless.
No, what it implies is that women put more effort into their work before making a (large) pull requests, not wanting it to be rejected. While on the other hand men are more willing to write some good/bad/mediocre code and risk failing.
First off they trumpet the fact that they discovered that women's merge acceptances were higher than mens. It's only when they sliced the one hundred thousands of accounts for "gender confirmation" that they decided that bias existed because success rates went from 72% to 64% - The error deviation of that alone should cover the spread.
Secondly the sample rating is awful - They compare TWO MILLION male checkins to ONE HUNDRED THOUSAND female checkins without any criteria for context, quality, need or style... just "quantity" and say that because the PERCENTAGE RATES FOR ACCEPTANCE are "higher" it must mean the women programmers are "Better" when comparing 2 sample sets with 20x the difference of checkins as they're all EQUAL.
Sorry. That's BS.
This is not science, this is propaganda statistics and poor statistics at that but I'm sure they made full use of their government funding to study gender issues in STEM fields.
There's no need for comparative statistics for men vs women, which leave you trying to control for all sorts of nebulous factors like how nicely they make requests, or how the genders might code differently.
All you have to do is take a bunch of coders (men or women, doesn't matter), and have them submit a bunch of code online using a male persona, using a female persona, and anonymously (or at least gender-neutral). Then compare acceptance rate for each individual. That neatly eliminates all other factors since you're comparing the same individual to himself or herself.
What message did the submitter (and posting editor) hope to convey?
Women are better coders than men?
Men are evil?
What are ./ readers supposed to do with this "information"?
Summary says their requests are "less likely to serve an immediate project need"... this is important, indicating they may have failed to control for "problem difficulty" in the study.
If something is an immediate project need, it's likely to get more attention but also likely to be more difficult. After all, if it were easy, it'd already be done.
If something is not an immediate project need, it's more likely to be easier, some random improvement that they dreamed up, a safe space to code where there won't be much contention, too much difficulty, and since it's a part of the project fewer people care about, getting it accepted is easier.
A pull request is less likely to be accepted if it's attempting to solve a difficult, important problem and the solution isn't absolutely top-notch.
A pull request is more likely to be accepted if it's solving a low-importance non-problem and the solution is just adequate.
I'd like to see a study of the ratio of M/F pull requests that are merely additive (a tip off from the article being they are larger) versus that which boldly improves and replaces existing lines of code.
The magnitude of the bias reported isn't alarmingly high so some of the things you suggest and others might be reasonable to consider as origins of the difference.
However, the change of the acceptance rate histogram from uni-modal to bi-modal when the gender is known for a woman seems to be much stronger evidence of gender bias.
The bottom axis of the histogram is rate of code rejections for an individual, and the left axis is the number of individuals with that rejection rate. When gender is not known both men and women have dominantly high acceptance rates tailing off towards low accpetance rates. However when gender is know a sharp second peak at the 90% rejection rate shows up on the women's histogram but not the men.
Thus I think what this study shows is that for the most part women work on code in ways that produces code more likely to be accepted. The fact that it tends to be longer and not something on the bug list may make their submissions different (more substantial infrastructure not defect fixes might be one interpretation). So I'm not inclined to conclude much from that. But the bimodality seems to be evidence of a strong gender bias among a small number of open source projects.
Some drink at the fountain of knowledge. Others just gargle.
I think he is saying that women are more pragmatic and stick with programming if they realistically have the aptitude and interest in it. Men may have more reasons to be in programming that aren't related to either one, so some of them produce crap. I don't think this reflects poorly on men or women and seems to match my experiences in the work environment.
love is just extroverted narcissism
The interesting thing the study actually found was that pull-request acceptance rates dropped for BOTH males and females when the gender of the requester could be inferred from their username or avatar picture. In some categories that rate dropped more for males, and in others the rate dropped more for females.
But they ignored the drop in rates for males and considered only the drop in rates for females when jumping to their conclusion of "gender bias".
I don't doubt there's bias against women, nor I want to mansplain the results at all. However I care deeply about good science and reliable facts. This article is not very good at showing clearly that this bias exists. Here are a few major problems with it:
1. Are the samples of women and men who post on GitHub representative of all open source programmers? I would think that women tend to contribute publicly less than man, and tend to disclose their gender less than men, and this probably biases the sample. The article doesn't attempt to analyze this, it merely assumes their sample is adequate.
2. The article says that both men and women get less pull requests accepted when their gender is identifiable, although this affects more women than men. The article does not compare the two values explicitly (in fact, it does not give the value for men at all), and it doesn't attempt to explain this effect -- it could well be that there's a confounding factor they haven't considered other then gender bias
3. Both men and women can accept and deny pull requests. Surely the gender of who accepts or denies the pull request is a factor that needs to be analyzed before the conclusion can be "there is gender discrimination"?
My Stack Overflow user
Ok great hypothesis equating outsider women to the ultimate evil on the internet.
Now, have you done a study and got evidence for your hypothesis or is nothing more than a fond notion of yours?
SJW n. One who posts facts.
Why is this poorly-researched inflammatory crap on Slashdot again?
Is someone looking to siphon yet more funding away from gender-neutral coding projects and into more "X for women" programmes?
"Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
The study has not shown what the submission or the study says it shows. What it shows is that when separated into two groups, women who self identify as women and those who don't, the two groups have their submissions pulled at different rates.
There is nothing in the study to show that the two groups are comparable in their ability to code. Another way to look at the numbers in the study would be to say that women who self identify as women are not as good at coding as those who don't. Both statements are equally invalid, the study doesn't show either.
"Grab them by the pussy" -- President of the United States of America
You might want to call guiness about that chip on your shoulder, you're approaching record territory.
We should start dealing in those black-market beagles.
After reading the summary and the summary of the article, I first thought that women's code is generally better:
Our results suggest that although women on GitHub may be more competent overall (...)
The trick is women on GitHub. In the conclusions they quote another study:
Another explanation is self-selection bias: the average woman in open source may be better prepared than the average man, which is supported by the finding that women in open source are more likely to hold Master’s and PhD degrees
Good job at pointing that out. If we are comparing PhDs to less educated people, it's expected that they have better code. That also makes the gender bias against women's code look even worst.
The only problem I have with the article is in the type of submissions part (programming vs non-programming). From the article:
For instance, changes to HTML could be more likely to be accepted than changes to C code, and if women are more likely to change HTML, this may explain our results.
The authors address the first part (they beat men in most languages, but to different degrees), and they didn't address the second: if .css and .json are 90% of their contributions, this would be the reason of their overall higher acceptance rate being statistically significant, this also could be another explanation to the rejection rate on outsider women vs outsider men, the only case with discrimination.
Both outsider women and outsider men have much lower acceptance rate than gender-neutral outsiders, but this is the only case where women supposedly are discriminated. IMHO, they should show the raw numbers (the confidence intervals show that they might not be that much), and check the languages that are in this group. If it's mostly .txt, .podspec, .m, .xml, languages where men statistically "beat" or might "beat" women, then the conclusion of bias could be wrong or exaggerated. It's just one hypothesis that I hope they do address when peer reviewing, just to rule out any chances.
As has already been explained in comments further up, a pull request is how changes to code get merged back into the master repository when using Git for revision control.
Anyone can download the code and make changes which they then commit locally. Then instead of 'pushing' their changes back into the original version, they issue a request to the maintainer to 'pull' their changes in. It is up the maintainer of the original repository to decide if they want the changes in their code or not.
The whole premise seems to be accepted pull requests = accepted developers. I mean they say:
"To what extent does gender bias exist among people who judge GitHub pull requests?
To answer this question, we approached the problem by examining whether men and women are equally likely to have their pull requests accepted on GitHub, then investigated why differences might exist."
The authors note that women are more likely to submit pull requests that aren't tied to existing open issues. They seem to conclude that this reinforces the idea that women have the best track records, that these requests are the hardest to get accepted.
"Thus, if women more often submit pull requests that address an immediate need and this is enough to improve acceptance rates, we would expect that these same requests are more often linked to issues."
I interpret that totally the other way. The paper equates getting a pull request accepted with being accepted, that's just not how (in my experience) development works. If you submit a patch for some feature add that only you've thought of, and it conflicts with nothing else, it's easy for a maintainer to accept. A patch for a known, open issue is much more likely to have regression considerations, and compete with other patches. If five people all submit a patch for one issue, odds are good at least four of them are going to be rejected. It's kind of like measuring an employee's productivity by how many lines of code they write. Experienced developers see that as largely silly.
Note in the graph at the top of page 15 that having an obviously-gendered profile hurt the acceptance rates of men even more than it hurt the acceptance rates of women. This completely undermines any conclusion that women are being discriminated against in pull request acceptance.
Check out my women's designer clothing store.
It could be also interpreted as people that don't want to self identify on the internet being more competent into making those pull requests than the ones that do.
First statistical conclusion in the article is faulty: the significance is based on a chi square with df = 3,064,667. Every difference is significant with a df that high. The second statistical conclusion has the same error: significant, but the difference here is marginal. These people should really think if the underlying data truly only represents a difference in gender and all other possible variables are identical.
But a large part of the article focuses on arguments like "they feel dejected" while in reality the numbers hardly differ. Not only that, they are even in the women's favor, even on the first request. How can you then complain about feelings of dejection or abandoning because of "an unreasonably aggressive argument style" (as if women are by definition incapable of that)? No, it's just clutching at straws because they have to write an article.
But it's the final graph that is the nail in the coffin of this article: even with their self-chosen statistics, there is no difference in acceptance rate for men and women when gender is known (although "known" is too strong a word), even in the outsider category. They then phrase it like this: "There is a similar drop for men, but the effect is not as strong" while not having even the cheapest statistical argument to support it. That's the best they can come up.
So the conclusion of this article should be: women have a slight advantage in pull requests on github. The rest is FUD.
Every part of this looks like a goal-seeking or bias-confirming study with an already-presumed conclusion.
Not sure, but that's not how science is supposed to work. That might fly for global warming studies though.
Twitter supports and protects racists - by smearing their critics with the "Hate Speech" label.
- Reverse-discrimination against men? Rejected, per the observation that there is evidence of discrimination against women when gender is identified.
Study CLAIMS no evidence of gender bias (while concentrating on bias towards or against women only) when gender is identifiable (from name, photo or profile on GitHub).
Study shows bias in interpretation of data though.
It ignores error bars in graphs (no real numbers are shown, just percentage graphs) AND paints a simplistic "more-pulls-4-women-except-when-identified" image.
Study's graphs on the other hand show NO BIAS towards women or men.
https://peerj.com/preprints/17...
First of all, when COMPLETE IDENTITY (and thus quality of work, not just gender) is known - there is NO SIGNIFICANT DIFFERENCE between male and female "pulls".
In "insider" cases where identity is known, supposedly gender-positive (you can tell who's male or female) IDs have a slight pro-female bias (~87.5% female vs. ~86% male), with barely existent error bars for males and tiny bars for females.
I keep using tildes, cause all that is presented are graphs - no percentages or real numbers are shown for this case, you have to eyeball them.
Where supposedly gender-neutral IDs are used, there is a tiny pro-male bias (~88% +/- ~1% female vs. ~88.5% +/- ~0.7% male).
I.e. There is NO pro- or contra-, male or female, bias when COMPLETE IDENTITY is known.
ON THE OTHER HAND...
In "outsider" cases where gender and identity is supposedly unknown (though clearly identifiable through email vs social network profiles comparison - which is where they got their data and what they are basing their study on)...
There is a slight pro-female bias in supposed gender-neutral IDs (~72% +/- ~1.5% female vs. ~69% +/- ~1% male) and a tiny pro-male bias in supposed gender-positive IDs (~62.3% +/- ~0.7% female vs. ~63% +/- ~0.0something% male).
Again, these are eyeballed values.
Study lists female gender neutral percentage in "outsider" cases as 71.8% and female gender identifiable percentage as 62.5% - claiming it as proof of bias against women who identify as such.
These are the only values presented in numerical form in this hypothesis.
I.e. There is NO pro- or contra-, male or female, bias when full identity is (supposedly) unknown either, but study tries to claim the opposite by ignoring own findings.
In other words, while looking for a hypothesis to explain their findings of bias, they accidentally took the gender-bias hypothesis they found behind the shed, and controlled it by putting a bullet in its head.
Then, not noticing that said finding of bias is dead, they kept on beating it, claiming it's alive and highly agile.
Kinda like in the dead parrot sketch, only here the salesmen really do believe that the parrot is just stunned and pining for fjords.
Mit der Dummheit kämpfen Götter selbst vergebens
What the study actually says is:
Now, when you actually look at the bar chart in Figure 5, you'll see that even at the 95% significance level (already a weak measure), there are no significant differences between male and female acceptance rates for "outsider" submitters. The only statistically significant difference in the graph actually is for insiders. And for insiders, the conclusion is the opposite: gender-neutral submissions are accepted at a equal rate, while women are favored when the gender of the submitter is known. Therefore, the only actual statistical evidence of gender bias is a bias in favor of accepting submission by women who are project insiders; but even that evidence is weak and the difference is tiny.
The biggest difference Figure 5 shows is in the acceptance rate between gender-neutral outside submitters and gendered outside submitters, and that difference is huge compared to all the other differences. That observation alone invalidates all the other conclusions of the paper, because it shows that the "gender-neutral" population is very different from the "gendered" population. Given that those populations are so different, even if there were statistically significant differences between the genders within those two populations, you couldn't conclude anything from them since the populations themselves are different in unknown ways.
Overall, the paper does not support the conclusion that there is gender bias against women. In fact, the paper doesn't show gender bias in favor of submissions by women either. Most likely, women (on average) simply have slightly different interaction styles and submission strategies, just like women (on average) have slightly different interaction styles and behaviors in other contexts.
It seems reasonable to assume, that women write code with a quality quite similiar to men's code. So getting more pull requests accepted may be related to being women (bias). But when they make their gender more important than their code, they annoy people with "hey there, I AM A WOMEN" and get less pull requests accepted.