Slashdot Mirror

← Back to Stories (view on slashdot.org)

Researchers Secretly Deployed A Bot That Submitted Bug-Fixing Pull Requests (medium.com)

Posted by EditorDavid on from the willing-to-commit dept.

An anonymous reader quotes Martin Monperrus, a professor of software at Stockholm's KTH Royal Institute of Technology: Repairnator is a bot. It constantly monitors software bugs discovered during continuous integration of open-source software and tries to fix them automatically. If it succeeds to synthesize a valid patch, Repairnator proposes the patch to the human developers, disguised under a fake human identity. To date, Repairnator has been able to produce 5 patches that were accepted by the human developers and permanently merged in the code base...

It analyzes bugs and produces patches, in the same way as human developers involved in software maintenance activities. This idea of a program repair bot is disruptive, because today humans are responsible for fixing bugs. In others words, we are talking about a bot meant to (partially) replace human developers for tedious tasks.... [F]or a patch to be human-competitive 1) the bot has to synthesize the patch faster than the human developer 2) the patch has to be judged good-enough by the human developer and permanently merged in the code base.... We believe that Repairnator prefigures a certain future of software development, where bots and humans will smoothly collaborate and even cooperate on software artifacts.
Their fake identity was a software engineer named Luc Esape, with a profile picture that "looks like a junior developer, eager to make open-source contributions... humans tend to have a priori biases against machines, and are more tolerant to errors if the contribution comes from a human peer. In the context of program repair, this means that developers may put the bar higher on the quality of the patch, if they know that the patch comes from a bot."

The researchers proudly published the approving comments on their merged patches -- although a conundrum arose when repairnator submitted a patch for Eclipse Ditto, only to be told that "We can only accept pull-requests which come from users who signed the Eclipse Foundation Contributor License Agreement."

"We were puzzled because a bot cannot physically or morally sign a license agreement and is probably not entitled to do so. Who owns the intellectual property and responsibility of a bot contribution: the robot operator, the bot implementer or the repair algorithm designer?"

87 comments

  1. Github sucks and is for stupid LIBERAL SJW by Anonymous Coward · · Score: 0

    that's right - sourceforge should be Your Choice

  2. Who owns a bot's intellectual property? by Anonymous Coward · · Score: 3, Insightful

    Who owns the intellectual property and responsibility of a bot contribution: the robot operator, the bot implementer or the repair algorithm designer?

    easy one: nobody

    Copyright applies to creative works. A machine produced work is not creative, since any similar machine could and would produce it.

    1. Re:Who owns a bot's intellectual property? by darkain · · Score: 3, Insightful

      This is great... in theory... Until it is actually tried in court, and they possibly rule a different way.

    2. Re:Who owns a bot's intellectual property? by Megol · · Score: 2

      One could argue that designer and/or programmer of the algorithm is providing the creative input. If the algorithm is based on machine learning it becomes a bit more difficult, can the selection of training data be considered creative?

    3. Re:Who owns a bot's intellectual property? by ShanghaiBill · · Score: 1

      One could argue that designer and/or programmer of the algorithm is providing the creative input.

      One could argue that, but there is very little legal precedent to support it. Once an algorithm is making something it is no longer "creative" in a copyrightable sense. A copyright to a software tool does not entitle the owner to copyright for the product of that tool, unless human created components such as libraries are included.

    4. Re:Who owns a bot's intellectual property? by Spamalope · · Score: 1

      The first multi-national company who decided they don't want anyone else to be able to use it.

    5. Re:Who owns a bot's intellectual property? by Anonymous Coward · · Score: 0

      Why does this remind me of the monkey selfie copyright lawsuit?

    6. Re:Who owns a bot's intellectual property? by helpfulcorn · · Score: 1

      My thought is that perhaps one could argue a something akin to: if one writes or even uses a program to generate art in a random way, they could likely claim (IANAL) it's their creation, certainly sell it as their own and possibly sue someone who copies it in an unauthorised way. This may work especially well as an argument due to a common lack of understanding amongst lawyers, government, etc.

      As a thought though, I wouldn't try to argue that it'd belong to the creator of the program since anyone can run it and then suddenly that creator would own an insane amount of code.

    7. Re:Who owns a bot's intellectual property? by Anonymous Coward · · Score: 0

      One could argue that designer and/or programmer of the algorithm is providing the creative input. If the algorithm is based on machine learning it becomes a bit more difficult, can the selection of training data be considered creative?

      One could also argue that the programmer who wrote the bug provided the creative input.
      Do you use an IDE with tab-completion? Who owns the code? The IDE-writer or you?
      I bet most characters were added by the IDE.

    8. Re:Who owns a bot's intellectual property? by Anonymous Coward · · Score: 1

      Mod up.
      What do you think compilers do when you set options like 'Bounds Checking' and 'Type checking'.

      During Y2K - remember that?, dozens of skillful edit macros written in REXX modified millions of lines of COBOL code. We did not call it AI. We did not cal it a bot. The OpenBSD people wrote macros to fix sloppy coding and string length issues - sometime Microsoft does not claim yet.

      There were cross-translators that converted COBOL to C and other languages.
      Borland Turbo C and Foxpro set some standards not seen today. Elegant efficiency and small code that would pass first year university.

      It seems to me todays code does not check function return codes, and shitheads will not code other in a CASE statement because that causes 'work'. I cant comment on an opertaing system looking at a string declared as a variable - and thinking 'I will execute it because it looks like a URL!' That never happened in Cobol of Fortran.

    9. Re: Who owns a bot's intellectual property? by Anonymous Coward · · Score: 0

      It's obvious that the operator owns the output and can sign the contributor agreement and then use the bot. If the work turns out to be not eligible for copyright at all it doesn't matter since it makes that part of the contributor agreement moot, but still valid. Just like using an IDE to do your refactoring.

  3. There's a lesson in this article. by Anonymous Coward · · Score: 4, Insightful


    During Expedition #1, whose results are presented in details in [7], Repairnator has analyzed 11,523 builds with test failures. For 3,551 of them (30.82%), Repairnator was able to locally reproduce the test failure. Out of 3,551 repair attempts, Repairnator found 15 patches that could make the CI build pass.

    Translation: Repairinator was able to fix .4% of the bugs it saw.

    A program repair bot is an artificial agent that tries to synthesize source code patches. It analyzes bugs and produces patches, in the same way as human developers involved in software maintenance activities. This idea of a program repair bot is disruptive, because today humans are responsible for fixing bugs. In others words, we are talking about a bot meant to (partially) replace human developers for tedious tasks.

    Instead of stating "Our goal is to enhance the performance of programmers" because that is what tools do; there are tons of businesses with sub-optimal solutions to their business process. Instead we use intentionally menacing speech.


    Their fake identity was a software engineer named Luc Esape, with a profile picture that "looks like a junior developer, eager to make open-source contributions... humans tend to have a priori biases against machines, and are more tolerant to errors if the contribution comes from a human peer. In the context of program repair, this means that developers may put the bar higher on the quality of the patch, if they know that the patch comes from a bot."

    Translation: We spent a fair amount of time lieing to people, justifying a means to an ends, not realizing lieing to people might cause them to not believe anything we say.

    Sounds like this guy is soon to be unemployed.

    1. Re: There's a lesson in this article. by Anonymous Coward · · Score: 0

      FYI It's spelt lying

    2. Re:There's a lesson in this article. by gweihir · · Score: 1

      Hahahahaha, nice! This basically shows that automation is actually incapable of tackling this problem. It probably wasted more human time with the 10 bad patches than it saved by producing the 5 that got accepted (but are not necessarily good).

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    3. Re:There's a lesson in this article. by ShanghaiBill · · Score: 4, Insightful

      This basically shows that automation is actually incapable of tackling this problem.

      Not really. If a company spends $10M per year fixing bugs, and this tool fixes 0.4% of them, then it just saved $40,000. Not bad for a free automated tool.

      Also, the hard part of fixing bugs is not writing the patch, but replicating, locating and isolating the problem. If you can say "Here is a bug, here is how to replicate it, and it is occurring in THIS function", then that is a big help, and this tool was able to do that for over 30% of the bugs. That is huge.

      It probably wasted more human time with the 10 bad patches than it saved by producing the 5

      Not necessarily. If the code looks like it has a bug, then it likely needs to be refactored to make it more readable, even if there is no actual bug. It is not enough for code to be correct, it also should be clear so it can be read and maintained.

    4. Re: There's a lesson in this article. by Anonymous Coward · · Score: 0

      FYI: it's spelled "spelled"

    5. Re:There's a lesson in this article. by citizenr · · Score: 1

      Translation: Repairinator was able to fix .4% of the bugs it saw.

      "fix"
      their bot commented out parts of the code that generated null pointer exceptions, its like commenting out half of Win95 kernel and calling it a fix.

      --
      Who logs in to gdm? Not I, said the duck.
    6. Re:There's a lesson in this article. by gweihir · · Score: 0

      You must be a "manager", as you have completely omitted all cost for the tool installation, the review of what it gives and the cost when it screws up.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    7. Re: There's a lesson in this article. by Anonymous Coward · · Score: 0

      -- commenting out half of Win95 kernel and calling it a fix.

      I'd call it a good start...

    8. Re:There's a lesson in this article. by Anonymous Coward · · Score: 0

      One more important detail: every single one of these "patches" required actual human review and most of them were found to be bad. This bot creates piles of completely useless and unnecessary work for humans. It doesn't solve any problem.

    9. Re:There's a lesson in this article. by Anonymous Coward · · Score: 0

      Laziness and fraud, with cut-n-paste high in the negligence list.

      I wrote a bot to check the changes I was supposed to automatically? approve passed muster.
      Wrong project, wrong cycle, wrong version, wrong target , wrong dates - well you did copy the last one, and the classic - no backout or recovery strategy. My brain held the names of trustworthy devs, and the ones whose changes followed defective procedures and careless practices.

      Zillions of Change Control macros and approval macros have been built.
      Nobody complains when some 'efficient worker' approved changes bite the dust.

    10. Re: There's a lesson in this article. by Anonymous Coward · · Score: 0

      You're a moran.

      Appropriate capcha: jested

    11. Re:There's a lesson in this article. by Anonymous Coward · · Score: 0

      Of course, an intern could have fixed twice as many of the off-by-1 and null pointer bugs that this tool found.

    12. Re:There's a lesson in this article. by Anonymous Coward · · Score: 0

      Who understands the code the bot wrote? Who can maintain it? How does it solve the problem better, as opposed to just passing a test? Code is written by people for people for a reason.

    13. Re:There's a lesson in this article. by Anonymous Coward · · Score: 0

      Sounds like this guy is soon to be unemployed.

      So you don't think that inexplicably strong emotional reactions to things are more likely to result in loss of employment?

      Have there been any completely inexplicable events in your career which you would like to share with us? I think this might be an enlightening and also highly entertaining line of discussion. It's just a guess!

    14. Re:There's a lesson in this article. by Anonymous Coward · · Score: 0

      "then it just saved $40,000"
      I don't think more than a few companies spend $10M a year fixing bugs. And you're missing the point. Most of these fixes are rudimentary, You need to think of it in developer time.
      Difficult issues to solve can take weeks. The simple issues are solved in minutes.

  4. Programmers are obsolete by Anonymous Coward · · Score: 0

    And software developers claim they are immune to automation. Hah. Like generational blacksmiths insisting tools could never create themselves.

    1. Re:Programmers are obsolete by dbrueck · · Score: 2

      How often is that claim really made? And if ever, how often is it actually made by developers? The only time I've ever heard that claim put forward is like you did: right before shooting it down.

      Automation is part of what makes development fun (there's a certain thrill to replacing something tedious with a "machine" that does it for you).

    2. Re:Programmers are obsolete by Anonymous Coward · · Score: 0

      Back in the early 1980s a program called "The Last One" was released which claimed the same thing.

      35 years later, we still have programmers.

    3. Re:Programmers are obsolete by mrbester · · Score: 1

      I don't recall when hammers could make hammers by themselves. Did they only appear in a Pink Floyd video?

      --
      "Wait. Something's happening. It's opening up! My God, it's full of apricots!"
    4. Re: Programmers are obsolete by Zero__Kelvin · · Score: 1

      He didn't say it in the right way, but he meant that he believes machines will program themselves one day.

      --
      Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
    5. Re:Programmers are obsolete by Anonymous Coward · · Score: 0

      Automation is part of what makes development fun (there's a certain thrill to replacing something tedious with a "machine" that does it for you).

      Absolutely. I automate any task which I have already manually completed at least 3 times - to gain a proper understanding of the processes and inputs & outputs - and which are likely to be repeated in the future. All of my data analytics and data science projects are automated at each stage albeit primarily to support reproducibility.

    6. Re:Programmers are obsolete by gweihir · · Score: 1

      Most developers are at risk of losing their jobs if anybody realizes how bad they actually are. Automation would just be the thing that shows that, not the thing that replaces them. Good developers will not get replaced until we have working strong AI, which is not happening any time soon. (A senior member of the IBM Watson team put it as "certainly not in the next 50 years" to me.)

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    7. Re: Programmers are obsolete by dbrueck · · Score: 1

      Eh, he's claiming that some broad segment of "developers" claim something and that this is evidence they are wrong.

    8. Re:Programmers are obsolete by Anonymous Coward · · Score: 0

      SLASHDOT EDITOR SUCKS, WIPED MY ORIGINAL COMMENT.

      So no more content for this site.
      ernulo nn nianeweaf lnerf aslni rgfnlirgsen;lnligrnilrg snilregsnilerg nils nil;s ni;lsgern;lgn;ilgr;elng nl;isgnlern;gasnsne n srggr s

    9. Re:Programmers are obsolete by Anonymous Coward · · Score: 0

      How often is that claim really made? And if ever, how often is it actually made by developers? The only time I've ever heard that claim put forward is like you did: right before shooting it down.

      Automation is part of what makes development fun (there's a certain thrill to replacing something tedious with a "machine" that does it for you).

      Frequently here on slashdot whenever people speak of automation costing jobs. Inevitably some developer will claim that they won't be the ones being ousted. The meme continues as that lot being replaced deserves their fate in some form for failing to do something that can't be automated, like doing the automation. Thus an explicit followed by an implicit assertion.

      And no, I am not claiming some broad segment has done this, only that this is an attitude which exists among all people that believe they can't be replaced due to the nature of their work, and specifically here with programmers given an example.

    10. Re:Programmers are obsolete by HiThere · · Score: 1

      Yeah, I remember that. It could handle choosing columns from a table if you told it which column ahead of time.

      That doesn't prove this is the same kind of thing, but PR flacks aren't any more moderate now than then, so it could well be.

      --

      I think we've pushed this "anyone can grow up to be president" thing too far.
    11. Re:Programmers are obsolete by dbrueck · · Score: 1

      How often is that claim really made? And if ever, how often is it actually made by developers? The only time I've ever heard that claim put forward is like you did: right before shooting it down.

      Automation is part of what makes development fun (there's a certain thrill to replacing something tedious with a "machine" that does it for you).

      Frequently here on slashdot whenever people speak of automation costing jobs. Inevitably some developer will claim that they won't be the ones being ousted.

      Are you sure? For kicks I checked a few stories (https://hardware.slashdot.org/story/18/04/24/2333259/a-study-finds-half-of-jobs-are-vulnerable-to-automation, https://hardware.slashdot.org/..., and https://slashdot.org/story/18/...) and didn't find really anything at all to suggest that "software developers claim they are immune to automation".

      In fact, if comments on those articles is any indicator, on slashdot it's typically the software developers who are in the "anything can be automated" camp. And really that makes sense - who is more likely to see where automation can go, the people who have decades of first hand knowledge and experience of progress in automation (software developers) or people who have relatively little direct experience with automation in their day to day job (a lawyer or an author maybe)?

      Maybe you had a prior bad experience with a deluded dev or two, but I think what you said was overly broad - or maybe my anecdotal experience is just the exact opposite of your anecdotal experience. ;-) Have a good one!

    12. Re:Programmers are obsolete by Anonymous Coward · · Score: 0

      There's a certain kind of manager who, instead of focusing on taking care of their people, motivating them, and trying to innovate at a cadence and push the industry foward, decides instead to nickel and dime their staff.

      These managers are soon surrounded by the three stooges, and like benny hill, slapstick ensues and their view of the world becomes ever more grim because they are on a sinking ship.

      And then they post on slashdot; "Gee Software developers aren't so great, are they? I guess even they can automate themselves".

      The reason we had a french revolution was because the people were so divided from the concept of self-respect or dignity they actually thought it a good idea to harass and torture the royalty and march them and their servants heads down the street on pikes, thinking somehow this ritual would bring them prosperity. It took them the better part of a decade to realize what they really needed to be doing.

      Please don't be daft like this.

    13. Re: Programmers are obsolete by Anonymous Coward · · Score: 0

      Wow. You think the reason there was a French revolution was ONLY because people didn't have any sense of dignity and wanted to be nasty to the royalty? You can't think of any other reasons that would have caused a revolution? Who is being daft?

  5. Copyright by Anonymous Coward · · Score: 0

    Code written by bots is not copyrightable because it wasn't made by a human. i.e. public domain.

    1. Re: Copyright by Anonymous Coward · · Score: 0

      What about computer generated graphics in Hollywood movies? Do you think we're allowed to copy those parts because a computer made them or do you think the operator of the computer, who purchased it, purchased or developed the software, and paid for the electric bill and the salaries of the operators... Should have some property rights to the result of their investments?

  6. Disruptive? by stephanruby · · Score: 4, Interesting

    If the author was so confident in his bot, he would have attached his own name to it instead of making up a fake name for it.

    Also, I don't see why he thinks his idea is so novel, static analysis, for instance, can suggest solutions if you want. And if you're too lazy to double-check the work yourself and let someone else do it for you, that's not a great discovery, that's just laziness.

    1. Re:Disruptive? by Anonymous Coward · · Score: 0

      Take this to the next level. Make the PR fix some bug, but are complex enough that they introduce backdoors. Using a bot allows you to do this in an automated way, and probability says it would only be a matter of time before someone merges in your malicious code and people updated to it. Maybe this will mean the death of PRs?

      Captcha: milkmaid
      (LOL, no evil maid as in evil maid attack?)

    2. Re:Disruptive? by slacka · · Score: 1

      Exactly, for example static analysis is great at catching copy/paste errors and will often show you the suggested fix. If I write a script that takes these results and submits a pull request, it's no different than if I manually take those actions. I,the script author would take credit, not Coverity or PVS Studio. This guys is out to lunch, thinking his tool is sentient and has legal rights.

    3. Re:Disruptive? by jameson · · Score: 1

      I'm not aware of any static analysis that can factor in unit tests (as this work does) to decide what does and what does not make a suitable patch. Note that these patches are about dynamic semantics, not just about name or type analysis failures (which might indeed be easy to fix purely by using static analysis).

      The reason for not attaching his name (or the name of the students working on the project) is to minimise bias. A patch coming in from a widely-published and highly experienced formerly-INRIA-now-KTH researcher might be viewed very differently than a patch coming in from a noob.

      The reason for `not just checking the patches themselves' is again to reduce bias, though you'll note that they did a manual sanity check every time before they submitted the patch. That is, they only submitted patches that they hoped wouldn't waste the devleopers' time after submission. Evaluating purely by manual validation means that the judgment of whether the result is useful relies purely on the honesty of the researcher, since most publication venues don't leave enough space for researchers to submit source code (nor sufficient time for the reviewers to analyse the source code and patches in detail, unless it's all pretty small and can be squeezed in with the rest of what one must describe in the typical 6-12 pages (double-column) or up-to-25 pages (single-column).

    4. Re:Disruptive? by Anonymous Coward · · Score: 0

      If the author was so confident in his bot, he would have attached his own name to it instead of making up a fake name for it.

      Also, I don't see why he thinks his idea is so novel, static analysis, for instance, can suggest solutions if you want. And if you're too lazy to double-check the work yourself and let someone else do it for you, that's not a great discovery, that's just laziness.

      If autistic shit-coders had any confidence in the garbage they churn out to keep capitalism chugging along, they'd take responsibility for their mistakes and accept liability.

      It's safe to assume that you will never do this. So why don't you just fuck off and die?

  7. Faster? by dbrueck · · Score: 2

    > 1) the bot has to synthesize the patch faster than the human developer

    I guess this is ok if it's faster in the sense of "the bot has to fix the bug faster than the human developer gets around to it", but in general I don't think this speed requirement needs to be that strict. If it's some latent bug waiting to hit us in production, for example, I don't care if the bot is slowly poring over code for days on end and brings a bug to my attention. Likewise, in any large project there are always tons of bugs that might not be that hard to fix, but it really is just a matter of getting around to them, so having some bot take a crack at them could be a good thing.

    1. Re:Faster? by Anonymous Coward · · Score: 0

      This.

      The problem isn't the bugs that I can find and fix. If I am aware of them and it takes me a week to fix then so be it.
      Worst case scenario is that I inform the customer that the bug exists and what the cost of fixing it is.

      The big problem is bugs that I don't know about. They will eventually pop up at the end user and the only thing I can answer truthfully is that I will look into it.
      If a bot can scan the code and give me a list of a things I should look closer at before shipping then that would be pretty damn helpful.
      If I get that list after shipping then that is sub-optimal but still infinitely better than waiting for the end-user to run into problems.
      Being able to respond to issues with an estimate of when the issue will be fixed avoids so many social problems that pops up with technical problems.

  8. How many were rejected? by Gravis+Zero · · Score: 4, Insightful

    What I'm reading is that yes, it made 5 patches that were accepted but the more important question is how many patches did it make total? If it made 800 patches and only 5 were accepted, that's kind of a problem.

    Also, there is good reason to distrust robotic submissions: there is no cognitive reasoning in generating patches. This means that it could very well make things worse rather than making them better. Sure, it could make your project build but it could also create an innocuous bug that breaks the code's functionality in the process which is likely to take even more time to correct because in addition to fixing the problem you also have to find it. Build failures already tell you where the problem exists.

    --
    Anons need not reply. Questions end with a question mark.
    1. Re:How many were rejected? by Anonymous Coward · · Score: 2, Informative

      What I'm reading is that yes, it made 5 patches that were accepted but the more important question is how many patches did it make total? If it made 800 patches and only 5 were accepted, that's kind of a problem.

      It made 15 patches, of which 5 were accepted and 10 were rejected.

      However, it tried to make 3,551 patches and only succeeded in 15.

    2. Re:How many were rejected? by TFlan91 · · Score: 3, Informative

      However, it identified 3,551 bugs, was able to automatically fix 15, and notified a human of the rest.

      FTFY.

      In that case, this sounds totally worth it. That's 3,551 bugs that QA or the client don't have to run into.

    3. Re:How many were rejected? by Raenex · · Score: 2

      In that case, this sounds totally worth it. That's 3,551 bugs that QA or the client don't have to run into.

      No, those were failures via automated tests. Meaning they were already known about. All this program did was provide a handful of fixes after trawling through thousands of known failures. Big whoop.

    4. Re:How many were rejected? by Anonymous Coward · · Score: 1

      In fact of those 15 patches, none of them got accepted or were found of low quality (i.e. not valid patches at all). They call that "expedition #1".
      Then they run it again a bit later, expedition #2, and got 5 patches accepted. They don't declare the same comparative figures as for the first run.

      Also they claim their system can fix 30 bugs a day. I wonder how they got to that number.

      All in all it's all hog wash. Interesting yes, yet still hog wash.

    5. Re:How many were rejected? by Anonymous Coward · · Score: 0

      All this program did was provide a handful of fixes after trawling through thousands of known failures. Big whoop.

      That is a pretty big whoop if those 3,551 files didn't require human interaction.
      If it automatically sorts through millions of files for possible bugs and is able to find a handful of things a human should look closer at then it is a great tool.
      If it can suggest fixes for a few of them, good, but the big deal is finding potential bugs, fixing them is usually not that hard.

    6. Re:How many were rejected? by nasch · · Score: 1

      it could also create an innocuous bug

      So could a human, and it happens all the time. That's what testing (automated and manual) is for.

      Build failures already tell you where the problem exists.

      This bot was fixing build failures?

    7. Re:How many were rejected? by Gravis+Zero · · Score: 1

      So could a human, and it happens all the time. That's what testing (automated and manual) is for.

      True enough but there is no cognitive design or debugging occurring, just code mutation, rebuilding and testing. In six months (and thousands of attempts later) it managed to make 15 patches, five of which where accepted. Some (if not all) of the five that were accepted needed to be modified as well.

      This bot was fixing build failures?

      I got that part wrong. From what I read, it's fixing unit testing failures.

      --
      Anons need not reply. Questions end with a question mark.
    8. Re:How many were rejected? by nasch · · Score: 1

      From what I read, it's fixing unit testing failures.

      Well there you go. If the unit tests all pass after the fix, and test coverage is acceptable, and there's a QA program to test the kind of thing unit tests don't cover, I don't see the issue. It's true that the entity that created the bug fix didn't understand what it was doing, but a human (the person who accepted it) presumably did understand it, and it passes all the tests that would be required of a bug fix from a human. Either the testing is adequate, or it isn't. If it is, then we can be assured the bug fix was adequately tested. If it isn't, you're rolling the dice whether the fix was from a human or a bot.

    9. Re:How many were rejected? by Gravis+Zero · · Score: 1

      Either the testing is adequate, or it isn't. If it is, then we can be assured the bug fix was adequately tested. If it isn't, you're rolling the dice whether the fix was from a human or a bot.

      You are correct. It's worth noting that it attempted to fix unit tests on 3000+ projects over six months, so it may be that poorly designed unit tests are what enabled it to "succeed" in making a patch for 15 projects.

      --
      Anons need not reply. Questions end with a question mark.
  9. Not disruptive technology... by Anonymous Coward · · Score: 0

    The bug-finding abilities of this software can be transferred to other software development tools, such as the compiler. BTW the tools are always being improved (so nothing new here).
    Just one more thing... It is a bad idea to automatically fix programming errors without checking the surrounding code. It is also stupid to program, then have the compiler find your errors... A good programmer is better than this.

  10. The robot operator owns property & responsibil by Anonymous Coward · · Score: 1

    How is this even a question?

    If you go and apply image filters to pictures on Flickr using SoftwareA... that doesn't mean SoftwareA is responsible for what you did or now owns what you produced, nor do SoftwareA's developers.

  11. Unnecessarily complex by skoskav · · Score: 1

    A more pragmatic approach for a project may be to run static code analysis on each commit in order to highlight potential bugs and vulnerabilities. Creating a bot that waits for tests to fail and swoop in with a patch based on such analysis then seems like a mostly redundant addition.

  12. Very bad idea by gweihir · · Score: 1

    In ordinary patch submission, there are two instances with actual intelligence and understanding: The patch creator and the maintainer. Here, there is only one: The maintainer. This violated the 4-eye principle. If the maintainer makes a mistake, the most stupid (in a non-obvious way) code makes it into the software.

    Automated tools should never be used to decide anything. They should always only provide input to a human expert that knows exactly how the input was created and that there is no intelligence in that mechanism.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    1. Re:Very bad idea by Anonymous Coward · · Score: 0

      The worst part of this whole thing is that if you look at the patches, you'll see that one of the four links doesn't exist anymore, and 3/4 of the others just guard against NPE's by adding an if-statement.

      Unfortunately, the last one listed also adds a this qualifier that would be a red-flag to any competent maintainer, because the patch would actually fail if there's a local variable by that name in the same scope (which could easily happen by accident in the future). Let's hope that the maintainer removed the this qualifier when merging the patch.

  13. Impersonating a bot by Morgaine · · Score: 1

    stephanruby wrote:

    If the author was so confident in his bot, he would have attached his own name to it instead of making up a fake name for it.

    It would be unethical for the human to impersonate a bot.

    What's more, the bot has no means to give the human legal authority to impersonate itself. Conundrum! :P

    [Oh dear. By the time I got to the end of this post, I began to realize that I was no longer quite so sure that I was joking.]

    --
    "The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
  14. Re:The robot operator owns property & responsi by Anonymous Coward · · Score: 0

    I agree, Repairnator is a tool. You don't pay the mechanic's tools to fix your car, you pay the mechanic.

  15. Look at the patches themselves by craighansen · · Score: 2

    I went to the trouble to look at the patches themselves, and they appeared to be lacking any documentation of which test case that was failing was fixed. If the bot did a better job of documenting the rationale for the patch, perhaps they'd get a better acceptance rate. (And by the way, the acceptance rate wasn't reported in the article, only that 5 patches were accepted over a 6 month period. - Was that 5 out of 6, or 5 out of 100?)
    Otherwise, I'd think that a program-generated patch, that indicated that it fixed a failing test case and didn't cause any additional failures in the test suite, and clearly indicated that it wasn't claiming proprietary rights to the patch would be generally welcome. I don't see the need for any secrecy.

  16. Researchers are obsolete by PolygamousRanchKid+ · · Score: 2

    Researchers Secretly Deployed A Bot That Submitted Bug-Fixing Pull Requests

    AI Bots Secretly Deployed A Researcher That Submitted This Research Paper

    --
    Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
  17. So what? Human reviewers a terrible at their job. by Anonymous Coward · · Score: 0

    Literally. Fake research papers have a way of making it in journals even after peer review. It's not clear what the quality of the patch was.

  18. hackers worldwide just got a great idea by RhettLivingston · · Score: 1

    How long will it be now before they make bots to submit patch requests that insert vulnerabilities - likely under the guise of fixing a true vulnerability and just making a mistake in doing so?

    1. Re:hackers worldwide just got a great idea by Anonymous Coward · · Score: 0

      Got this same idea. For fun: fix an actual problem, but inject a vulnerability at the same time. Make it obscure enough, and it will likely get accepted. If you called the bot something like "skynet" or something, it would be perfect.

  19. Self patch by GrahamJ · · Score: 1

    Pointing such a bot at its own codebase is how skynet happens.

  20. What Ethics Committee let this pass? by Pimpy · · Score: 1

    Following up on the links from the article, it's clear that the professor in charge only went out and informed those who accepted the pull requests more than 10 months after the fact of the situation, under the guise of "full disclosure" (and these are of course for links he has only linked directly, we have no information about the failed pull requests). While one of the areas they wished to focus on was inherent discrimination against bots, he appears to have missed the point that automation, while able to provide simple fixes to obvious problems (predominately in error handling flows), they have a terrible track record of understanding nuance or interaction subtleties during run-time. Coccinelle went through a lot of this with the Linux Kernel as well and is something that ultimately received a great deal of acceptance as it matured - although in this case, patches were generated automatically and submitted by humans. I wonder then what the objective of this research is, to show that software developers are sceptical of contributions from automated tools that follow semantic rules more than logical reasoning, or that by hiding behind a fake identity representative of a demographic that is more likely to submit trivial changes you are able to hoodwink the developers into accepting your submissions? Both get an ethics fail, and it's not clear that there's anything even novel to show for it.

  21. Improvement over regular code analysis tools? by Dirk+Becher · · Score: 1

    How can this perform better than regular code analysis tools who already inform you of nullptr accesses, unused variables etc. ?

  22. c6gunner's impersonating me & lying by Anonymous Coward · · Score: 0

    See subject: c6gunner's name on this post as submitter yet signed "APK" (me) https://linux.slashdot.org/com...

    I never say hosts cure Spectre/Meltdown OR it'd be on the Start64.com download page for it & I do NO MacOS X one!

    * Too bad I publicly DESTROYED you c6gunner both here https://tech.slashdot.org/comm... + here https://tech.slashdot.org/comm... vs. your LIES/LIBEL + on hosts' technicals https://tech.slashdot.org/comm...

    APK

    P.S.=> Due to you TRYING TO PUT WORDS IN MY MOUTH I NEVER SAID!: DNS in kernel? I never said that! I did get the BLOW YOU AWAY ON IT 4th link above! Hosts is tunable NATIVE resolver driven by tcpip.sys IN kernelmode in Windows + kernelmode diskcache in Linux + hosts resolves faster, safer vs. slow remote DNS that can be DOWN or DNS poisoned & NO DNS TRACKING for fav sites I use @ TOP of hosts for speed/security vs. DNS poisoning & rest are BLOCKED (who cares how 'fast' getting to them - I NEVER INTEND TO GET TO THEM, TTL does rest))

  23. Malwarebytes' Steven Burn audited my code by Anonymous Coward · · Score: 0

    See subject: He BOTH hosts + RECOMMENDS it -> forum.hosts-file.net/viewtopic.php?f=5&t=4290

    My code's also VIRUS PROOF (self-checking mathematically down to 1 byte change) upmodded @ CODING FOR DEFCON on /. in 2005 as a good method http://it.slashdot.org/comment...

    I won't give my work away so "OpenSORES" thieves can use it (or abuse it, see EFast/Google Chrome).

    APK

    P.S.=> Dozens of REGISTERED /. users (+ 100,000 users worldwide) disagree with you https://it.slashdot.org/commen...

    Hosts results STOPPING MALWARE = UNDENIABLE https://science.slashdot.org/c...

    SECURITY PROS agree hosts work https://it.slashdot.org/commen... for MORE speed AND security & they do so NATIVELY for less (resources consumed, less moving parts for exploit vs. SECURITY ISSUE RIDDLED DNS/AntiVirus (which slow you down vs. local hosts in RAM cached) ... apk

  24. Well, this one patch may be the reason by Anonymous Coward · · Score: 0

    All the others seems to be just wrapping assignments in null checks.
    But this one shows clearly it works differently:
    https://github.com/eclipse/ditto/pull/151/commits/7ab96e373ecc55e1b942bfff4c9550c8ba5e584d

    Still, just one example is a fairly thin proof of usefulness.