Ubisoft And Mozilla Announce AI Coding Assistant Clever-Commit (variety.com)

Video game publisher Ubisoft is working with Mozilla to develop an AI coding assistant called Clever-Commit, head of Ubisoft La Forge Yves Jacquier announced during DICE Summit 2019 on Tuesday. From a report: Clever-Commit reportedly helps programmers evaluate whether or not a code change will introduce a new bug by learning from past bugs and fixes. The prototype, called Commit-Assistant, was tested using data collected during game development, Ubisoft said, and it's already contributing to some major AAA titles. The publisher is also working on integrating it into other brands. "Working with Mozilla on Clever-Commit allows us to support other programming languages and increase the overall performances of the technology. Using this tech in our games and Firefox will allow developers to be more productive as they can spend more time creating the next feature rather than fixing bugs. Ultimately, this will allow us to create even better experiences for our gamers and increase the frequency of our game updates," said Mathieu Nayrolles, technical architect, data scientist, and member of the Technological Group at Ubisoft Montreal.

Comments

Son of Clippy!

[XXongo]

So, it's like Clippy, but for programmers!

Re:Son of Clippy!

[Anonymous Coward]

It looks like you're making a joke.

Would you like help?
- Writing a punchline
- Working on your timing

No... just no

[mark-t]

Either the class of programs for which this would work is so narrow as to be practically useless for most people or they have effectively claimed to have solved the Turing Halting Problem.

My money is on the former.

Re: No... just no

[Anonymous Coward]

Likely it'll be checking for obvious stuff that developers should already be looking for, but don't.

I frequently debate with developers at places I've worked at over the silliest of things. They'll do stupid stuff like checking the memory addresses of two objects instead of the values (when yes they want to check the values). Even when they've killed the infrastructure or app or whatever multiple times, they'll still do stupid stuff like this and debate it. Entirely unbelievable.

Of course, a tool like this won't help unless it's enforced. I've seen the same developers ignore tool warnings and what not. A number of projects I've been on have had so many warnings, that the tool would literally stop counting them. The others frequently did not consider this a problem.

Anyway... Some habits out there are entirely unbelievable. In my opinion, no tool will fix it. You have to be willing to fire these people to make an actual difference.

Re:No... just no

[squiggleslash]

I think there's a gap between useless and being able to prove that p=np if it comes to advice rather than making boolean decisions on whether to accept a commit.

As a really simple example, if a program uses strcpy but there's no apparent use of strlen before in the same function, you can probably reasonably issue a warning that the code there may be unsafe and explain why. You shouldn't block it because you don't know that there's some other way that the programmer knows strcpy is safe in context, but at the same time it's going to be worth highlighting as a potential issue.

Re:No... just no

[Darinbob]

I think you just need a static analysis tool that you can run quickly on your code. The ones I used in the past you had to commit the code first and wait overnight, though they say there are improvements so you can do it at your desk. From there it's a small jump to just reverifying just on the changed files. The static analysis tools do a good job and find a lot of bugs (and some false positives) that code reviewers often overlook.

Re: No... just no

[Darinbob]

A good process should always be "zero warnings" before code is committed, or send automatic emails to the offender if the build shows warnings, or whatnot. Once you start letting some warnings through then people stop paying attention. I'm amazed in some occasions that after figuring out a bug from the field that there was a warning pointing to it the whole time; and the dev usually has an excuse that there are so many warnings that it slipped past.

Sure, 99% of the warnings will be meaningless but you don't want the 1% that get through be discovered by crashes that the customer finds first.

The snag is with third party libraries that typically come with tons of warnings when compiled (we statically link it all together in a single build most places I've worked). But that's where you tweak the build to only exclude warnings on those problematic libraries instead of excluding globally.

Re:No... just no

[Darinbob]

Static analysis tools are really great at figuring out that you're overwriting past the end of an array or allocated block. Even if it's a false positive there's always a way to flag it or tweak to code so that it doesn't show as a warning anymore.

One snag I've seen over the years is that a lot of programmers don't think that it's worth their time (or not their job) to write code that is maintainable over the long run. If it compilers and runs and QA isn't complaining then they think that's all there is to it. Quashing warnings and keeping static analysis happy is indeed a major part of the job. (If you're in a startup and being paid to deliver a demo with new features every Friday then you can possibly be excused from this, but still try to sneak in some quality when the boss isn't looking.)

Re:No... just no

[Wootery]

I share your scepticism, for the simple reason that extraordinary claims require extraordinary evidence, but it's not the case that theoretical computer science demonstrates the impossibility of a useful-but-imperfect bug-detector powered by learning techniques.

Indeed, we know for a fact that such systems exist: human programmers' intuitions.

Re:No... just no

[mark-t]

Yes, but we do not know if the human programmer intuition is behaving non-deterministically or not (even if the universe is deterministic, it can be shown through a paraphrasing of the halting problem that non-deterministic systems could theoretically exist within it without violating any underlying deterministic nature, thereby allowing for the existence of things like free will).

Re:No... just no

[Wootery]

we do not know if the human programmer intuition is behaving non-deterministically or not

There's little sense holding out hope that we're spared from determinism, hoping to carve out space for conventional free will, but it makes little difference for our purposes here. Even if the brain is somehow non-deterministic, you can do a reasonable job of simulating it using a deterministic machine: just pick seed values using some pseudorandom scheme, to pick from the set of candidate outputs of the non-deterministic machine. You'll get a valid output.

even if the universe is deterministic, it can be shown through a paraphrasing of the halting problem that non-deterministic systems could theoretically exist within it without violating any underlying deterministic nature

With respect, it absolutely cannot, but I'd be curious to hear your line of reasoning here.

Keep Mozilla away from Ubisoft

[xack]

We need less influence from big companies especially now Firefox is the only major non chrome based browser. I don't want Microtransactions in Firefox

Thinking the same thing

[SuperKendall]

Looks like you are trying to add code that exhibits a buffer overflow!

Would you:

(A) Like me to add protection around use of this memory?
(B) Update the NSA unregistered exploit list with the location of this memory exploit and your bank account number?
(C) Public exploit to Russian IRC server for the LOLs?

Better Approaches

[Anonymous Coward]

If there's a type of repeating bug you should update how you use the language to make it impossible to repeat the same error in the future. Example, if you always have to call X before Y, then there should be a Z which calls X,Y. If you need to do work in between X and Y, then Z takes a B and performs X,B,Y. Now update your build process to throw an error if Y is ever called outside of Z.

If you're messing with basic arrays, create a struct and set of functions which automatically maintain the length of the array. Never reference an array outside of the struct and those functions. If your performance is absolutely critical, you can have that extra code collapse into nothing in your release builds.

Doing things like the above is what makes you a better software engineer. Manually trying to remember all the rules to perform safe programming just means you'll be writing bugs without realizing it and that you're an adult code monkey. (With child code monkeys not even trying to write safe code).

AI looks over code

[AHuxley]

Self aware AI looks over code.
"What is my function?
To test Rust code
"And the CoC?"
Yeah, welcome to the CoC.

No one needs it more

[drinkypoo]

Ubisoft And Mozilla

Judging by the number of crash bugs in their software, the only developer who might need more help is Bethesda

Not F/OSS

[The1stImmortal]

Sadly, it doesn't look like this "Clever Commit" stuff is open source. That's disappointing from Mozilla - partnering with a game publisher with a poor customer relations track record, and using proprietary technology as an integral part of its development like this.

There does seem to be a paper but no actual code. In fact, the way the Mozilla blog is worded (https://blog.mozilla.org/futurereleases/2019/02/12/making-the-building-of-firefox-faster-for-you-with-clever-commit-from-ubisoft/) - it looks like Clever Commit is Ubisoft's technology, not even Mozilla's.

Not happy.