Slashdot Mirror
Researchers Secretly Deployed A Bot That Submitted Bug-Fixing Pull Requests (medium.com)
An anonymous reader quotes Martin Monperrus, a professor of software at Stockholm's KTH Royal Institute of Technology:
Repairnator is a bot. It constantly monitors software bugs discovered during continuous integration of open-source software and tries to fix them automatically. If it succeeds to synthesize a valid patch, Repairnator proposes the patch to the human developers, disguised under a fake human identity. To date, Repairnator has been able to produce 5 patches that were accepted by the human developers and permanently merged in the code base...
It analyzes bugs and produces patches, in the same way as human developers involved in software maintenance activities. This idea of a program repair bot is disruptive, because today humans are responsible for fixing bugs. In others words, we are talking about a bot meant to (partially) replace human developers for tedious tasks.... [F]or a patch to be human-competitive 1) the bot has to synthesize the patch faster than the human developer 2) the patch has to be judged good-enough by the human developer and permanently merged in the code base.... We believe that Repairnator prefigures a certain future of software development, where bots and humans will smoothly collaborate and even cooperate on software artifacts.
Their fake identity was a software engineer named Luc Esape, with a profile picture that "looks like a junior developer, eager to make open-source contributions... humans tend to have a priori biases against machines, and are more tolerant to errors if the contribution comes from a human peer. In the context of program repair, this means that developers may put the bar higher on the quality of the patch, if they know that the patch comes from a bot."
The researchers proudly published the approving comments on their merged patches -- although a conundrum arose when repairnator submitted a patch for Eclipse Ditto, only to be told that "We can only accept pull-requests which come from users who signed the Eclipse Foundation Contributor License Agreement."
"We were puzzled because a bot cannot physically or morally sign a license agreement and is probably not entitled to do so. Who owns the intellectual property and responsibility of a bot contribution: the robot operator, the bot implementer or the repair algorithm designer?"
It analyzes bugs and produces patches, in the same way as human developers involved in software maintenance activities. This idea of a program repair bot is disruptive, because today humans are responsible for fixing bugs. In others words, we are talking about a bot meant to (partially) replace human developers for tedious tasks.... [F]or a patch to be human-competitive 1) the bot has to synthesize the patch faster than the human developer 2) the patch has to be judged good-enough by the human developer and permanently merged in the code base.... We believe that Repairnator prefigures a certain future of software development, where bots and humans will smoothly collaborate and even cooperate on software artifacts.
Their fake identity was a software engineer named Luc Esape, with a profile picture that "looks like a junior developer, eager to make open-source contributions... humans tend to have a priori biases against machines, and are more tolerant to errors if the contribution comes from a human peer. In the context of program repair, this means that developers may put the bar higher on the quality of the patch, if they know that the patch comes from a bot."
The researchers proudly published the approving comments on their merged patches -- although a conundrum arose when repairnator submitted a patch for Eclipse Ditto, only to be told that "We can only accept pull-requests which come from users who signed the Eclipse Foundation Contributor License Agreement."
"We were puzzled because a bot cannot physically or morally sign a license agreement and is probably not entitled to do so. Who owns the intellectual property and responsibility of a bot contribution: the robot operator, the bot implementer or the repair algorithm designer?"
easy one: nobody
Copyright applies to creative works. A machine produced work is not creative, since any similar machine could and would produce it.
During Expedition #1, whose results are presented in details in [7], Repairnator has analyzed 11,523 builds with test failures. For 3,551 of them (30.82%), Repairnator was able to locally reproduce the test failure. Out of 3,551 repair attempts, Repairnator found 15 patches that could make the CI build pass.
Translation: Repairinator was able to fix .4% of the bugs it saw.
A program repair bot is an artificial agent that tries to synthesize source code patches. It analyzes bugs and produces patches, in the same way as human developers involved in software maintenance activities. This idea of a program repair bot is disruptive, because today humans are responsible for fixing bugs. In others words, we are talking about a bot meant to (partially) replace human developers for tedious tasks.
Instead of stating "Our goal is to enhance the performance of programmers" because that is what tools do; there are tons of businesses with sub-optimal solutions to their business process. Instead we use intentionally menacing speech.
Their fake identity was a software engineer named Luc Esape, with a profile picture that "looks like a junior developer, eager to make open-source contributions... humans tend to have a priori biases against machines, and are more tolerant to errors if the contribution comes from a human peer. In the context of program repair, this means that developers may put the bar higher on the quality of the patch, if they know that the patch comes from a bot."
Translation: We spent a fair amount of time lieing to people, justifying a means to an ends, not realizing lieing to people might cause them to not believe anything we say.
Sounds like this guy is soon to be unemployed.
What I'm reading is that yes, it made 5 patches that were accepted but the more important question is how many patches did it make total? If it made 800 patches and only 5 were accepted, that's kind of a problem.
Also, there is good reason to distrust robotic submissions: there is no cognitive reasoning in generating patches. This means that it could very well make things worse rather than making them better. Sure, it could make your project build but it could also create an innocuous bug that breaks the code's functionality in the process which is likely to take even more time to correct because in addition to fixing the problem you also have to find it. Build failures already tell you where the problem exists.
Anons need not reply. Questions end with a question mark.