Which Programming Languages Are Most Prone to Bugs? (i-programmer.info)

An anonymous reader writes: The i-Programmer site revisits one of its top stories of 2017, about researchers who used data from GitHub for a large-scale empirical investigation into static typing versus dynamic typing. The team investigated 20 programming languages, using GitHub code repositories for the top 50 projects written in each language, examing 18 years of code involving 29,000 different developers, 1.57 million commits, and 564,625 bug fixes.

The results? "The languages with the strongest positive coefficients - meaning associated with a greater number of defect fixes are C++, C, and Objective-C, also PHP and Python. On the other hand, Clojure, Haskell, Ruby and Scala all have significant negative coefficients implying that these languages are less likely than average to result in defect fixing commits."

Or, in the researcher's words, "Language design does have a significant, but modest effect on software quality. Most notably, it does appear that disallowing type confusion is modestly better than allowing it, and among functional languages static typing is also somewhat better than dynamic typing."

Comments

In before Fractal of Bad Design

[Waccoon]

You already have to be a genius to understand functional languages, so of course those people make fewer mistakes.

I love it when functional fans insist it's more analogous to how the brain really thinks. That's why so few people can figure out how to do things that way.

Re: In before Fractal of Bad Design

[_merlin]

Pascal and ANSI C are very similar, but pre-ANSI C is a completely different beast, far more similar to BCPL. In fact, ANSI C could almost be described as Pascal with C syntax.

Pre-ANSI C didn't have prototypes - it assumed any undeclared name was an external function. It didn't automatically convert int to long if the function expected it, etc. - you had to explicitly cast. You had to be careful to cast results of functions correctly, too. All it had was a set of rules for how argument types were stacked, and it was up to you not to pass something a function wasn't expecting. This is closer to assembly language programming than Pascal or ANSI C.

Re: In before Fractal of Bad Design

[Anne Thwacks]

Assembly to C was quite easy as well.

Which is hardly surprising, since C is just PDP11 assembler tidied up a bit.

Which is interesting considering the PDP11 was "a hardware Fortran machine", and the i386 architecture is a close copy of the PDP11 - the 286 even copied the early (variable page size) PDP11 memory management scheme that was the fashion before people figured out it was not really good for virtual memory, and then the 386 copied the later (fixed page size) PDP11 MM which is needed for virtual page based memory. The PDP11 was designed on the assumption that memory bandwidth was the bottleneck in throughput.

The fact that C has been successfully ported to almost all more recent processors is largely because the concepts of what a CPU is and does have been developed by people who grew up on the PDP11 architecture. RISC architecture may be different, but, in reality, the RISC architecture is mostly just used to simulate CISC anyway. RISC was designed on the assumption that instruction decode was the bottleneck - which has not really been the case since the 1980's - hence the failure of RISC to displace CISC.

I am writing this on a Sparc64 (RISC) machine - the story is more complex that I describe here.

2018

[AHuxley]

Rediscovers how great Ada would have been for the consumer.

Complexity

[Anonymous Coward]

Or could it be that the software written in C++ usually tends to be large complex software where performance is important along with various other complicating factors. While the software written in ruby for example tends to be simpler?

Sounds like this 'study' started with a conclusion already in mind.

Haskell and C++ programmers are different.

[shess]

Something the linked article didn't seem to address it that the population for each language will differ. The average Haskell programmer is going to be very different from the average C++ programmer, or, god forbid, the average Python programmer.

Also, while they did try to address problem domains, I don't think they addressed systemic issues. For historical reasons, there are many projects which use C or C++ simply because of what they need to interface with to get the job done. For instance, there simply aren't going to be that many browser projects which aren't written in C++.

Personally, I think the interesting take-home is not the difference between languages, it's how small the number of commits for security and memory issues was.

Re:Haskell and C++ programmers are different.

[serviscope_minor]

Also, while they did try to address problem domains, I don't think they addressed systemic issues.

I don't think they do: none of them have things like zero overhead abstractions, zero cost memory allocation and so on. And some of them (like go) lack the kind of abstractions present in many modern languages.

For instance, there simply aren't going to be that many browser projects which aren't written in C++.

Of the three remaining extant enignes: Firefox, Webkit/Blink and Edge and Trident all except firefox are written in C++. Firefox is partly Rust now.

Rust I think is one of the very very few languages aimed a the same problem domain as C++ by people who understand enough C++ to know what the problem domain was. Look for example at Pike's rants on GO and how was designed to replace C++ and didn't: many C++ programmers sikmmed the features and said something like "oh that'll make my program slowe, more verbose, buggier and harder to write". Rust on the other hand is the same machine model as C++ but with a very very different type system.

It's never going to replace C++ across the board that's for sure but it's proven capable of replacing C++ in a niche where formerly there were no contenders.

Conclusions only valid on Open Source Projects?

[mykepredko]

This is an interesting study, but I don't know if the results can be extrapolated to include closed source software.

My problem with this is that I don't see any evidence of:
a) Projects in the study have a published project plan with somebody managing it at a high level (I would think the Linux Kernel could be thought of as having a plan with strong central management ). I tend to believe that projects in which multiple individuals (with varying levels of understanding of the software, the app's background and issues experienced during development) would be at a much lower quality level than something managed by a strong, continuous team - this doesn't seem to be a consideration when I RFTA (popularity of projects seems to be a bigger issue).
b) Different development tools used by different developers. In terms of the C/C++ typing issues, Windows software developed and built in Visual Studio, Eclipse Text Editor with MinGW or something like Komodo Edit with Cygwin and user written make files will identify different typing issues and may generate code that works differently, especially in regards to identifying and handling typing issues. I would like to know how many bug fixes are the result of something that isn't flagged and works fine on VS and doesn't work when built in MinGW, leading to a fix.
b.1) I'm not 100% sure of the methodology used in this study, but wouldn't a file that originally had tabs for indentation that an editor automatically changes it into spaces be misidentified as a "fix" if it's uploaded back into the repository? This is a combination of b) and c).
c) Different coding styles. I know of several Open Source projects in which a developer has re-formatted code simply because they don't think it's in the "correct" style and they have difficulty reading it resulting in them changing it so they can follow it better. To be fair, I'm sure a lot of us have done that because some people have very different and strongly felt ideas about how code should be formatted.
d) Lack of formal testing methodologies. I don't think many Open Source projects have strong, automated regression testing processes and methodologies before allowing a new release.
e) Difference in functional use of different languages. I would think that methods written in C, C++ and Objective C would be providing more low-level functionality than Clojure, Haskell or Scala. Ruby probably fits somewhere between the two groups.

Comments?

Python

[_merlin]

I know I'll get flamed for this, but Python is really error-prone in a particular area, and that's its ridiculously weak name resolution rules. In a language like C, Perl, or even PHP, names are resolved during the compile phase. The compiler knows which definition of a name is going to be used at any point. Python doesn't have this - when it runs across a name, it walks up the scope hierarchy looking for a candidate.

This means that code can run happily for months or even years, until it just crashes with an undefined name error. This could be because of a rarely-used code path with a typo in it, botched refactoring of a rarely-used code path, or a particular set of rare circumstances where a global name isn't set before the code gets to a certain place.

The usual response is that unit tests should catch this. But let's face it, 100% unit test coverage is pretty rare, particularly for the kind of fast turnaround stuff that Python's frequently used for. Also, unit testing isn't necessarily going to simulate a corner case where a global doesn't get set before code that uses it executes. It also makes refactoring more risky because there's no point where the compiler can tell you you're referencing a name that's no longer defined, or no longer has a certain method/field.

This is the kind of area where it's really useful if the compiler can help you, and Python's ridiculously weak name resolution rules make that completely impossible.