Slashdot Mirror
The Effect of Programming Language On Software Quality
HughPickens.com writes: Discussions whether a given programming language is "the right tool for the job" inevitably lead to debate. While some of these debates may appear to be tinged with an almost religious fervor, most people would agree that a programming language can impact not only the coding process, but also the properties of the resulting product. Now computer scientists at the University of California — Davis have published a study of the effect of programming languages on software quality (PDF) using a very large data set from GitHub. They analyzed 729 projects with 80 million SLOC by 29,000 authors and 1.5 million commits in 17 languages. The large sample size allowed them to use a mixed-methods approach, combining multiple regression modeling with visualization and text analytics, to study the effect of language features such as static vs. dynamic typing, strong vs. weak typing on software quality. By triangulating findings from different methods, and controlling for confounding effects such as team size, project size, and project history, they report that language design does have a significant, but modest effect on software quality.
Quoting: "Most notably, it does appear that strong typing is modestly better than weak typing, and among functional languages, static typing is also somewhat better than dynamic typing. We also find that functional languages are somewhat better than procedural languages. It is worth noting that these modest effects arising from language design are overwhelmingly dominated by the process factors such as project size, team size, and commit size. However, we hasten to caution the reader that even these modest effects might quite possibly be due to other, intangible process factors, e.g., the preference of certain personality types for functional, static and strongly typed languages."
Quoting: "Most notably, it does appear that strong typing is modestly better than weak typing, and among functional languages, static typing is also somewhat better than dynamic typing. We also find that functional languages are somewhat better than procedural languages. It is worth noting that these modest effects arising from language design are overwhelmingly dominated by the process factors such as project size, team size, and commit size. However, we hasten to caution the reader that even these modest effects might quite possibly be due to other, intangible process factors, e.g., the preference of certain personality types for functional, static and strongly typed languages."
e.g., the preference of certain personality types for functional, static and strongly typed languages.
My guess is that this has a bigger impact on most projects than actual features of a chosen language. I was thinking it the whole time I read the summary and then, sure enough, it's mentioned as a disclaimer at the end...
Creationist Textbook Stickers Declared Unconstitutional by CowboyNeal
It's clear that there are more factors here that need to be normalised out. For example, they found that the category that "had" the most performance bugs was the procedural, static, unmanaged memory category, i.e. C, C++ etc, far outstripping languages like ruby. To me, it's clear that that is caused by people using these languages actually caring about performance, while people using languages who's implementations are many orders of magnitude slower, don't really file (or fix) bugs about perf.
Almost no casual programer uses functional languages and do not tend to be used for large FOSS projects.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
For functional languages, the ideal is a language that supports both procedural and functional construct, so you can use either when it makes sense. Scala, and to a lesser extent C#, fit that bill quite nicely.
Dynamically typed languages are great in cases where you'd be writing all the test cases anyway. UI code is a good example. Its faster to write proper JavaScript unit tests. provided you have the necessary infrastructure, than to manually test click click click. Since you're writing all the tests anyway, then having a dynamic language has very little drawbacks.
The motivation behind this paper was probably the intuition that language choice greatly influences KISS. But frankly, I've never had a real project where language choice wasn't driven by external forces.
For a project starting in 1983, and expected to last 8 months, Basic seemed like a good idea. By 1995, it should have been obvious to everyone that a re-write in ANYTHING ELSE was justified (not that I would personally recommend using Perl to write a Basic interpreter to re-interpret the original Basic code or using Snobol4 to translate the Basic into Fortran).
I could have used C instead - the project would probably have taken a couple of weeks longer, but would have saved countless people years of grief. I have C programs from the 80's that compile on *BSD unchanged, and still work as intended*. It was a toss-up at the time.
My point is that the language choice may be influenced by incorrect information about the external world - because the external world is subject to massive change.
* I had to rewrite some C from the 70's cos they were written for Idris and all in capitals :-{ Fortran4 programs from the 70's may compile and run, but you certainly need to re-test them!
Yes its true: my lawn is written in Fortran, but my Mum's has an Ibjob border.
Sent from my ASR33 using ASCII
As for which language is your gateway language, it probably depends on what era you started programming in, too. My path was Microsoft BASIC => Assembly => Turbo Pascal => C => Perl => C++11, with some shell scripting and other goodies around the fringes. I've probably written more C than anything, but C++11 rules my future. Turbo Pascal was my short-lived gateway to C, where I then spent most of my career. I wrote some truly regrettable neophyte-programmer code in C there at the beginning, so really C was where I grew from a college-aged hacker to someone who can actually program. Now guess how old I am. ;-)
I guess for an analysis like this, you really need to limit yourself to people who consider themselves competent programmers. Those VB macro whizzes in accounting likely consider themselves accountants, not programmers. Likewise for the physicist with a pile of creaky MATLAB models.
BTW, I have to agree with you 100% on make and bash. I consider myself above average on make as compared to my coworkers, but that's an extremely low bar. And while I've done some crazy stuff in bash in the past, these days I'll hop over to perl for anything more than 10 - 20 lines, especially if I find too much 'sed' showing up or find myself wanting an actual data structure.
Program Intellivision!
Absolutely! They write:
"The major languages that contribute concurrency errors
from these classes are Go, C#, and Scala and their regression
coefficient is also statistically significant. These results confirm,
in general static languages produce more concurrency errors than
others. Among the dynamic languages, only Erlang is more
prone to concurrency errors. The regression analysis also shows
that projects written in dynamic languages like CoffeeScript,
TypeScript, Ruby, and Php have fewer concurrency errors"
Well, there isn't much concurrency to be had in TypeScript or CoffeeScript since they operate in single threaded environments, and it wouldn't surpise me if the same goes for the other ones. And saying that Erlan has problems with concurrency is... the cart before the horse.
I've used functional programming languages at university and in my career started off with strongly typed languages, but four years ago I retrained to Ruby and haven't looked back.
I'm a SCJP and a OCMJD and when I was working through the OCM assignment, I couldn't conceive of a language nicer than Java. My first encounter with dynamic languages was with Groovy. I was already a keen test-driven developer but at first I had to absolutely TDD the code 100% because I was so paranoid about this new language I was using. In this way I kept bugs out of the code by having a comprehensive suite of tests. Since moving to Ruby, I've found that TDD practices are more ingrained than with a language like Java so the lack of compiler errors doesn't bother me. Early on in my career code was deemed correct if it compiled, even if it wasn't tested, which has put me off relying on the compiler too much.
Python also uses hash tables everywhere, including for global and local variables and instance and class members.
The implementations of the hash table is extremely optimised.
There was an interesting discussion on usenet about someone who wanted to optimise a piece of Python code by writing a C++ module to do the heavy lifting. It was about adding C++ strings into a C++ hash map. The naive version was 10,000 times slower in C++.
Of course the naive function was copying strings many times since this is what the string class does when passing by value. After many posts they came up with a optimised version, it was about 100 lines of code (original was 10 lines), with lots of references passing and explicit and complex template instantiation of the hash map and its iterators. It was still 1.2 times slower than the Python implementation.
Relevant parts highlighted:
"The remaining coefficients are significant and either positive or negative. ... a greater number of defect fixes.
For those with positive coefficients we can expect that
the language is associated with
These languages include C, C++, JavaScript, Objective-C, Php, and Python.
The languages Clojure, Haskell, Ruby, Scala, and TypeScript, all have negative coefficients
implying that these languages are less likely than the average to result in defect fixing commits"
Isn't the real message here to choose static typing if you like fixing bugs?