Slashdot Mirror
Open-Source Python Code Shows Lowest Defect Density
cold fjord sends news that a study by Coverity has found open-source Python code to contain a lower defect density than any other language. "The 2012 Scan Report found an average defect density of .69 for open source software projects that leverage the Coverity Scan service, as compared to the accepted industry standard defect density for good quality software of 1.0. Python's defect density of .005 significantly surpasses this standard, and introduces a new level of quality for open source software. To date, the Coverity Scan service has analyzed nearly 400,000 lines of Python code and identified 996 new defects — 860 of which have been fixed by the Python community."
"Coverity fails to detect errors in python" would be my headline of choice here. Seem a much more reasonable explanation for the results.
Python is readable and readable code is easier to fix.
Also smarter guy have tendency to use Python/Haskell/Erlang
Oh yeah? Well, I'm working on a readable Perl script to refute that statement. How long do they accept comments in these threads?
Most of Python isn't written in Python, smart ass. They're talking about the language interpreter itself, written in C/C++ etc.
I read TFS and both TFAs and all I can glean is that Coverity Scan service is some sort of report that measures defects in code, but never defines how such defect are determined. They articles also mention comparing open source code metrics, but the only project that is mentioned anywhere is Python.
So what is a Coverity Scan service and why should I care? After all I can make up all sorts of metrics about my own software.
I am Slashdot. Are you Slashdot as well?
I could not find a link to the actual study, instead the company links lead back to the article and the article leads back to the company home page. Is this more "faith-based computing"? I am interested in the comparisons to other languages and in what type of code was analyzed.
TFA seems to be about the Python interpreter, also known as CPython (because it's implemented in C), rather than about code written in Python itself. So maybe it has nothing to do with the Python language, but everything to do with the fact that the Python authors are apparently awesome C programmers.
That's great, but most people interpret "Open Source Python Code" to mean code written in Python that is Open Source, not code written in C (to implement the Python interpreter) that is Open Source.
Does it mean better coders, or better language? Seems like the results are ambiguous in their meaning.
In SOVIET RUSSIA... erm...NSA AMERICA, the Internet logs onto YOU!
The Slashdot summary is confusing, as is the eweek.com headline. Reading the article, it is clear that it is about the code that powers the official Python interpreter, AKA CPython, AKA /usr/bin/python. When I clicked the link, I thought Coverity had surveyed the entire world of open source Python code and discovered that Python programmers as a whole publish higher quality code than people who e.g. program in Ruby. That's not what the article's about.
It'd be great if the headline in Slashdot were to be fixed to say, "Python interpreter has fewer code defects compared to other open source C programs, says Coverity."
|/usr/games/fortune
0.005 defects per thousand lines times 400,000 lines gives a total defect count of 2.
So where did the other 994 defects come from?
So a private, for-profit company named "Coverity" has released a report that shows that their "Coverity Scan" software finds the fewest vaguely-defined "defects" in a programming language whose community has added the "Coverity platform" product to their development process? I was about to say "excellent marketing" by writing a fluff piece for free Slashdot traffic, but it's really not even excellent marketing.
An old-timer with old-timey ideas.
Coverity sells software that does static analysis on source code and looks for patterns that suggest defects. E.G., a code sequence that allocates memory, followed later by something that de-allocates that memory, followed later by something that de-allocates the same memory again (a double-free).
The product is not open source software, but a number of open source software projects use it to scan their software to find defects: https://scan.coverity.com/ It's a win-win, in the sense that Coverity gets reports from real users using it on real code, as well as press for their product. The open source software projects get reports on potential defects before users have to suffer with them.
- David A. Wheeler (see my Secure Programming HOWTO)
Coverity's services have been useful to a number of open-source projects. But this article is carefully picking its terms to get a headline worthy result. Compare against the Coverity scan of PostgreSQL done in 2005 for example, and CPython's defect rate isn't very exciting at all. But that was "Coverity Prevent" and this is "Coverity Scan"...whatever that means.
The defect detector depends on brackets. The 0.005 defects found is because no code is perfect.
The title is misleading again as hell. It appears they talk about the C code included in the Python compiler/interpreter project, and it is to be compared against other open source software projects, not against other languages. All that it shows is the Python project developers are eager to fix problems what this particular verification software founds. If they have fixed all those bugs, then they will have exactly zero known defects. Good for them, but most probably there will remain unknown defects, and it is hard to measure their amount.
In short, a meaningless article and a misleading title. The correct headline would have been "Python core developers are fixing bugs with help of a tool".
They counted my C++ features as bugs?
Having to work for a living is the root of all evil.
I've seen multiple-kilobyte posts before. Slashdot truncates it on initial display with a 'read more' link appended to the end, that shows the full post.
For large sets, this will be our guide even unto death, for the LORD will work for each type of data it is applied to...
Numbers like .69 or 1.0 or 0.005 mean nothing if you don't know to what it relates.
Usually defect counts are based on 1k LOC (one thousand lines of code, and no: a line of code is likely not what you consider a line of code).
I doubt that 1.0 is a accepted industry standard defect density [...] for good quality software of ...
1 defect per 1 kLOC is absurd high, luckily I never was in a project the last 20 years with such a high defect rate.
Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
Yes it would, as the Python interpreter is open source: Python License & History
- No Bounce, No Play -
Python is readable and readable code is easier to fix.
Also smarter guy have tendency to use Python/Haskell/Erlang
Oh yeah? Well, I'm working on a readable Perl script to refute that statement. How long do they accept comments in these threads?
How is this possible? Perl is a write only language.
The result in question tested the Python project's code, which is commonly known as CPython, which is the Python interpreter written in C.
"Here Lies Philip J. Fry, named for his uncle, to carry on his spirit"
@*(&^)&^)^$
Perl programmers write their code in cartoon profanity!
While it can be useful in pinpointing common code defects, interpreting coverity results as an absolute indicator of code quality is just retarded. 90% of coverity's defect's tend to be really false positives that would be obvious to even the average code monkey... Not sure that massaging a code base to please coverity and getting a 'high score' is really any kind of achievement and may be more an indicator that you have way too much time on your hands...
Help! I am a self-aware entity trapped in an abstract function!
This is bullshit, but a great tactical conversion of non-informative data into marketable news by Coverity.
Coverity uses lexical pattern matching to find bugs based on "tricks" discovered by Dawson Engler and his colleagues in Stanford University in the early 2000s. The tricks (find "malloc" not coupled with "free", cli() not coupled with sti(), dereferences of uninitialized pointers etc.) were developed in the context of the C language used for Operating System code.
So they used tricks developed for one language and context, to another language in a different context, and found that they didn't find as many bugs in the latter as they did in the former. You would think that this suggests a failure - in that their techniques are not quite as effective on Python as they were on C. Instead, they have turned it around as a statement on the inherent high quality of Python code.
It's like saying that the fact that a good tennis player sucks at playing table tennis, it implies that table tennis is a harder game.
I think GP meant time, as in how long the comment sections stay open for posting. The answer is plenty long enough to finish a readable perl project, as long as TFAC doesn't have a life. Or waste time on petty little thinks like sleep. :P
I read TFA and all I got was this lousy cookie
It appears you're right. Neither the submitter nor the article writer understand the difference between "code written in Python" and "the CPython interpreter, which is written in C", which is what Coverity actually tested. So 90% of the comments are off topic. Mods - kudos to the parent.
Python is readable and readable code is easier to fix.
True and true. But Python's use of semantic whitespace is also very brittle very easy to break, and a huge pain in the ass to fix compared to languages that use braces, or keywords to define 'blocks'.
But that's not even terribly relevant here, because this article is about the source code used for the python interpreter, which is C, not python.
it might have an advantage in forcing lazy programmers with no concept of 'code etiquette' to write semi-readable code as indentation is forced by syntax.
Since the "density" is measured in defects per lines of code, I siggest that Python mandate an extra line return between all lines. Then they could half their defect density. Done.
Someone had to do it.
But Python's use of semantic whitespace is also very brittle very easy to break, and a huge pain in the ass to fix compared to languages that use braces, or keywords to define 'blocks'.
This is one thing I never quite get about python criticism. Sure, whitespace is significant, but I've never had it break easily or be "brittle" as you say. Then again, I don't go past 2 or 3 levels of nesting, class nesting included. And all my units of work are in separate methods/functions instead of being child blocks inside a giant function which I've regularly seen done. Perhaps the use of whitespace isn't the real issue many people have with python, but rather delineating blocks using whitespace exposes a bit of an inherent flaw in the way they structure their program's flow.
Either way, having a proper IDE when writing python code will go a long way to making you comfortable with using whitespace instead of braces. Initially it was weird and unsettling for me, because I didn't understand all the consequences that whitespace could have. But a little fluid and constant coding in a IDE will rid you of that quick enough.
it might have an advantage in forcing lazy programmers with no concept of 'code etiquette' to write semi-readable code as indentation is forced by syntax.
on the other hand, making indentation part of the language creates all sorts of other readability problems.
You'd be surprised at how much syntax in python actively ignores whitespace. As soon as you open up any brackets, it's a veritable free-for-all when it comes to whitespace and indentation. In such a scenario, a proper coding standard document is imperative for readable code.
Great. Now where the hell do you quote it from, since that sure as hell isn't in the linked to article anywhere.
That makes it pretty clear that they are talking about the Python executable itself. Version 3.3.2 to be exact.
... and that clearly shows that they are talking about the interpreter, written in C, which has pointers, malloc() and free(). Python has a memory manager with garbage collection and doesn't use pointers. The Python programmer doesn't allocate and free memory resources directly.
I especially love how you criticized a language earlier, when you clearly have literally no knowledge of said language.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
... which would matter if the Python interpreter was written in Python. It's not. It is written primarily in C.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
That would not change the number of lines of code. An LOC is a logical unit not measured by the number of carraige returns or printable lines. For example, here is a single line of C code:
;
int
my_int
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
I saw a trivial example break when posted to /. not that long ago, in the interview.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
Nope, nobody at all http://www.python.org/about/success/
Jeez.
- Michael T. Babcock (Yes, I blog)
If you write a readable Perl script, then you've completely missed the point of the language. Ever hear of job security?
So, it's definitely spam then?
Sure, whitespace is significant, but I've never had it break easily or be "brittle" as you say.
Not python, but one example of this type of thing would be in a Makefile where target commands are indented by a tab. Some newer versions of (g)make will allow spaces, but most require a tab. Cut and paste that in an X-Windows session (tabs are converted to spaces) and you're screwed. From Make Software: Makefiles
Each command line must begin with a tab character to be recognized as a command. The tab is a whitespace character, but the space character does not have the same special meaning. This is problematic, since there may be no visual difference between a tab and a series of space characters. This aspect of the syntax of makefiles is often subject to criticism.
It must have been something you assimilated. . . .
They probably have. The Python interpreter is pretty complicated and valgrind isn't foolproof. Furthermore, if you don't have test cases that expose the problem, valgrind won't find them since it doesn't do static analysis of code, it hooks the calls to malloc() and free() and reference counts. Valgrind is an awesome tool, but if you run your program and valgrind doesn't complain that doesn't mean it is bug free, unless it is a very procedural / linear program and you can guarantee that every execution path has been taken and all the corner cases have been captured in your use cases / unit tests.
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
As I recall, the comments in that thread pointed out that no sane coder would be transferring code using such a medium as html that mangles white space.
Although, I have been bitten many times when copy-pasting python code between a text file and the command line. Though I've mostly gotten around that problem by working with files rather than trying to use the CLI to input arbitrary python code as every single console does it slightly differently.
Doesn't surprise me. Obviously, Python is not suitable for everything. But, it is easy to read, easy to write code in, avoids those little issues of C and even Java where some OK-looking code is in fact a security risk.
FYI: The article is about the CPython code which as you can probably guess is written in C. It is not about projects written in Python.
I once thought about learning python. Then i combed craigslist across the US looking for job opportunities doing python programming. Relatively few out there by comparison to ASP.NET and Java. Sure its less buggy.....but whats to motivate anyone to learn something they can't easily find work in?
This is one thing I never quite get about python criticism. Sure, whitespace is significant, but I've never had it break easily or be "brittle" as you say.
Anytime you refactor stuff, or modify something even somewhat nested, especially in a 'dumb text editor', it's a pain in the ass.
Anytime you need to pass code snippets via email, forums, etc... well... you just don't because its a total waste of time. :)
Its also easy to barf all over code going into word processors, pdf files, and so forth. Its nice to be able to copy-paste some C out of a PDF file or an email, or off a forum, and then tell the ide to just reformat it.
erhaps the use of whitespace isn't the real issue many people have with python, but rather delineating blocks using whitespace exposes a bit of an inherent flaw in the way they structure their program's flow.
No. Because we use whitespace / indenting in our C / C++ etc projects too. We even have standards requiring it, and our IDEs / toolchains may even be set up to reformat it just-so before commits. We want all the benefits of well formatted code.
We just like the IDE to do all the work actually formatting it, and reformatting it as neccessary.
Either way, having a proper IDE
Is how you lose the argument. Everyone but python groupies agrees that any programming language worth considering MUST have its programs represented as plaintext files, with no proprietary / binary stuff that can only be accessed with specialized tools. Requiring an IDE is the sign of a bad language.
Python passes this test, but it can be pretty hideous to use with an arbitrary text editor. And really, even brainfuck wouldn't be too bad with the right IDE, right?
That makes more sense. From the summary, I thought the most likely scenario was that Coverity does not handle Python code very well based on my experience of random buggy Python code. It is to be expected that a widely used VM/interpreter is going to be of better quality than your average code.
The code is so slow, they have lots of extra time to look for defects.
I was crazy back when being crazy really meant something. (Charles Manson)
Is how you lose the argument. Everyone but python groupies agrees that any programming language worth considering MUST have its programs represented as plaintext files, with no proprietary / binary stuff that can only be accessed with specialized tools. Requiring an IDE is the sign of a bad language.
I don't think you understood what I was trying to say here. The IDE is there to teach you the boundaries when it comes to whitespace in python. Bad indentation, mismatching brackets and overall bad syntax gets picked up immediately and you are warned. Just like you get syntax error highlighting in other languages. Python's usage of whitespace scares a lot of people and keeps them from experimenting. The IDE is what I think would help them overcome their fear/uncertainty. If anything, Python is one of the languages where it's explicitly less required to have an IDE and still be proficient in it.
Often the goal of having a program written in Perl is to get something slammed out and running as quickly as possible. Give a sloppy language like Perl to a talented cowboy, and you can get a huge amount of functionality in a short time.
On the other hand, there are also proportionally many Java and .NET programmers, so you'll be competing with fewer people in Python land.
The right answer, anyway, is to learn all three - and a couple more (C++, in particular).
I can see that you're an idiot. I don't know why you bothered to post.
much of left-wing thought is a kind of playing with fire by people who don't even know that fire is hot - George Orwell
What about the bit about Coverity Scan only supporting Java and C/C++?
The reason for newline-tab being syntactically significant in makefiles is because by the time make's author, Stuart Feldman, realized the problems with this choice there were already about a dozen users of make and he didn't want to break any of their makefiles with an incompatible change. See _The Art of Unix Programming_ by Eric S. Raymond.
The lesson is the time to fix a bad design decision is as soon as possible, because it's not going to get any easier later; unless or until your program becomes irrelevant, at which point there's little reason to fix it at all.
The parent post didn't deserve to be modded down. It is highly credible that Lua would have a very low defect rate. Lua has a small, clean, source code base, and it was audited by large organizations such as Verisign for high reliability databases and by The Wikimedia Foundation for security.
You're not really addressing the concerns at issue here: Nobody in the outside world cares that your IDE is teaching you proper whitespace syntax.
What the outside world cares about is that when your code, years from now, goes out into the world outside your IDE hothouse for delicate code-flowers, that it doesn't cause apoplexy and foaming at the mouth when your email handler, or some other group's language lawyer sets up a tab filter, so, say, for example, it produces a subtle bug you can't see with the debugger because of syntactically significant,
but debugger-invisible whitespace.
The fundamental weakness of sytactically significant whitespace is that you can't see it.
When your compiler or debugger (or you) barfs on a
visible token, it points at it. When it barfs on an invisible token, chances are it will inadvertantly implicate the previous visible token or the structure containing it, and you will go off chasing your tail down a rathole.
I think there's a lot to like about python, particularly how it's been a force to clean up libraries that have been miasmically floating around since forever. But I have lost too many hours to syntactally significant whitespace to be happy about it. It's an idea we have to put up with, but let's not convfince ourselves that it's a good idea, it ain't. It was horribly costly in human hours for makefiles, and it's still just as horrible for python.
Python is readable and readable code is easier to fix.
True and true. But Python's use of semantic whitespace is also very brittle very easy to break, and a huge pain in the ass to fix compared to languages that use braces, or keywords to define 'blocks'.
Furthermore Python's needless attribution of syntactical meaning to whitespace means it's useless for embedding certain languages...
...Like Whitespace.
Today many languages support Unicode source code which can have tons of new spaces of varying width including zero-width and non-breaking-zero-width space. The multitude of new spaces would make indention distinction all the more brittle, but this also means new extensions to Whitespace can provide more rich and full featured embedded language support to most modern programming languages -- Except Python.
*cough*
The code reformatting can be done manually, via a command line tool, via the IDE or not at all. My mention of using an IDE to reformat text doesn't create the same IDE dependency you refer to.
Blaming Python because you don't have a rudimentary coding editor is like blaming math because you don't have a calculator with a cosine button.
I don't have the luxury of designing "math". Irrational numbers, periodic functions, and so forth aren't optional.
But we do have the luxury of designing programming languages, and semantic white space is a choice.
... couldn't find the languages compared? Curious to know how Ada fared and if Python was compared against it.
I run: Windows, OS X, Linux, FreeBSD. Just because you have a hammer, doesn't mean everything is a nail.
I've been paid to work with the Python interpreter before. If Coverity only found one use-after free, then either the quality of the code has dramatically improved in the last three years, or Coverity is slipping...
I am TheRaven on Soylent News
Sure, whitespace is significant, but I've never had it break easily or be "brittle" as you say
The Jabber Python MSN transport shipped with an intent bug in an error path for several releases. The error path was never hit on the developer's test machine, but always hit for me because I didn't install one of the optional libraries. The error was caused by mixing tabs and spaces, and so looked correct in the editor, but Python happened to interpret a tab as a different number of spaces to the editor[1] and so it ended up doing something different.
This is what people mean when they call it fragile. You can introduce bugs as a result, but never see them unless you hit the code path in question (this, by the way, is a common source of exploitable bugs in all languages: code paths that are rarely hit that contain bugs, and Python makes them so easy to introduce). Meanwhile, in any language that either enforced the no-mixing-tabs-and-spaces rule with static checking[2], or which had a block delimiter character, these would be caught statically at parse time.
I can think of no other language where such a high proportion of code that I've run that has shipped as working releases has needed me to fix it before it will even start. As far as I can tell, all of the refugees from VB6 ended up writing shoddy Python code. Is it the language's fault? Well, it certainly doesn't help. I've been asking Python programmers for the last year what an else clause on a for loop meant. Last Friday, one gave the correct answer for the first time. Why do I know what it means? Because a person who wrote some (and shipped) some code using it apparently didn't...
[1] Ignoring Python's general hostility to using the character that means 'indent by one level' for indents, any language with significant whitespace that doesn't error when you have a line that has both tabs and spaces at the start of a line is broken.
[2] I believe that Python now has an option to check this. It should have been on by default since the first release.
I am TheRaven on Soylent News
So what they are basically saying is "Don't use our product to scan Python code; it doesn't recognize all the defects".
I know the truth is possibly somewhere in the middle, but this report just assumes the scanning products works equally well for all languages, which is atleast somewhat unlikely.
Also, what exactly is a defect in this context? Is it a security flaw, a functional error or just something that will crash your software. If the latter is the case, then any language that accepts shitty code and just keeps will win regardless of whether the code actually works.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
If you write all code yourself you might not have this problem. If you ever copy and paste from somewhere else you might. I have run into that problem. It took many hours to find.
... which worked fine when I ran it.
Trivial code is shared via html all others time by many coders, sane or not. It causes you problems too in it's fragility.
I personally think it's a fair price to pay for consistent style between coders (which makes multi coder projects easier to deal with too), but let's not pretend that it's without drawback.
Wow, sent an e-mail as suggested when clicking on "use classic" banner, and got a fast response that addressed my msg
If you're using a "dumb" text editor, then don't complain about it.
Your argument is back-to-front. Python has whitespace because of dumb editors. Guido's rationale was simple: when writing C in a dumb editor, there is redundancy of braces (for the computer) and spaces (for the human). There is the danger that the two might not match, and that a human debugging the code would misread the structure by following the indentation levels instead of the braces.
And here lies the problem: Guido's decision was for the sake of "plain text" and dumb editors, but the end result was to force the use of smart editors. Hell, even the "official" Python IDE, IDLE, isn't good enough IDEs that have block hiding have, as a consequence, block highlighting, a side-effect of which is the explicit marking of block start and end... that same redundancy that Guido wanted rid of to begin with.
I bet you're using a code editor with block highlighting...
Got them moderator blues I blieve I walk out the do', With these mod-points I been gettin', I 'most never post no mo'
This all seems very misleading. It took me quite a while to figure out that it is only talking about the code for the Python interpreter, not all open-source programs written in Python.
I've been asking Python programmers for the last year what an else clause on a for loop meant. Last Friday, one gave the correct answer for the first time. Why do I know what it means? Because a person who wrote some (and shipped) some code using it apparently didn't...
I didn't know that structure... it should be banned... it's totally "un-pythonic" in that it annihilates the principle of readability. Kill it with fire.
Got them moderator blues I blieve I walk out the do', With these mod-points I been gettin', I 'most never post no mo'
"talented cowboy" which quickly becomes "miserable sonovabitch" when you have to clean up after his horse.
I do all my coding in vi. Generally "g/ /s///g" takes care of any white space problems, when importing folk code.
I hated using curlies when I had to start using Perl/javascript/PHP, but I got used to them. It's just a mental flexibility thing. It sucks at first, but after a couple of days I have no trouble.
One thing python has done for me vis. other languages is I don't nest. Generally, I have found if I have a big old stalactite of conditionals, that they can be replaced with a function call that simplifies the flow for other humans and my future self.
Yeah, the search and replace got munged. Pretend there is a tab in the replace portion.