Open Code Has Fewer Bugs
ganns.com writes "Reasoning, which sells automated software inspection services, scrutinized part of the code of Linux and five other operating systems, comparing the number and rate of programming defects. Specifically, Reasoning examined the TCP/IP stack and found fewer errors in Linux. 'The open-source implementation of TCP/IP in the Linux kernel clearly exhibits a higher code quality than commercial implementations in general-purpose operating systems,' the company said in a report released last week. Reasoning also compared the code with that used in two special-purpose networking products and found it superior to one of them."
But bugs are cool..does that make me a geek for using Redhat?
I guess it stands to Reasoning that more developers hammering on code leads to fewer bugs. :)
Companies such as Oracle and Microsoft typically sell binaries incomprehensible to humans rather than the comparatively understandable source code.
After seeing this, I think that statement is being a bit generous
To make laws that man cannot, and will not obey, serves to bring all law into contempt.
--E.C. Stanton
How about using a larger sample of code before making such bold statements. It's probably true that the code has fewer bugs, but when you abuse statistics it just makes things look dishonest.
Pope is catholic
Bears are found to sh*t in woods
Free cell phone tracking
Over time, successfull open source projects which address a particular issue will most likely have fewer bugs; just being open source doesn't mean fewer bugs (or better software). It just means that it has a better chance, if it survives, of being better software.
"What we have here, is a failure to communicate." - Cool Hand Luke
Open source has fewer bugs.
Bill Gates: bugs are cool.
Ergo, open source is not cool!
Really? But I thought most commercial OSes derived their TCP/IP stacks from BSD code in the first place. And since BSD is open-source, shouldn't these commercial OSes show roughly the same level of quality then? Or are they arguing that the Linux TCP/IP stack is superior to the BSD one?
Cheers,
Ian
Reasoning, which sells automated software inspection services, scrutinized part of the code of the Linux and five operating systems,
Including the Solaris, the Windows, the AIX, and the HP/UX.
I want to drag this out as long as possible. Bring me my protractor.
I find the fact that they did not say what OSes they compared to be very... suspect. What about Mac OS X, FreeBSD, and other open source OSes that have Open Source TCP/IP implementations in their kernels? Since they did not say what OSes are being used...
"Reasoning declined to disclose which operating systems it compared with Linux, but said two of the three general-purpose operating systems were versions of Unix."
How lame. For all we know, they could have tested the Amiga OS, Mac OS 9, Windows 3.1, A/UX, and NeXTStep! Other than this, the article is pretty vague and does not seem to give me much meat on the subject, nor a link to the study (you have to go through some forms and give up personal info to get it at www.reasoning.com).
Conglom-O: We Own You (TM).
The attitude I've seen in the corporate world is that open source products are made by amateurs and is therefore in some way not blessed by the magical corporate coding fairy which makes all the shit churned out by corporate code shops stink less. This attitude is arrogant and does not take into account the simple fact that all those people who got into programming just for the money tend not to work on open source products. When you've got code that is both written and reviewed by legions of people who love to code and who find good computer programs to be beautiful, you're going to get better code.
I'm trying to teach myself to set people on fire with my mind... Is it hot in here?
The more points of view you apply to solving a problem, the quicker, and better you'll solve it. The beauty of human reasoning isthat no two people will view the world in *exactly* the same way, therefore each one of their respective paths to the solution will be different...Travelling that path to one solution can, as we know, lead to other SOLUTIONS to other PROBLEMS.. The more heads that work, the more solutions discovered . . and so on..
Open Code Has Fewer Bugs
The study looked at a single part of an operating systems (TCP/IP stack) and then the posting made a very general claim about open source software. This is cheap engineering (a.k.a. bad science). Period. You need a much larger sample to make such a claim. A single data point is meaningless. In fact, I believe that code bugs are much more a function of programmer performance and code complexity than open vs. close source development model. Opening the code may have a positive impact, but it is not the major factor to consider. The last thing Open Source needs is this kind of marketing strategies...Severity of bugs is more important that number of bugs. I could have a program with one bug, doesn't work. And you can have two bugs, feature broken and memory leak, but it works. Who is better?
I also like the assumption in the title that because linux was found to have fewer bugs than some other OS's that open software in general has fewer bugs. Take a look at some of the bug lists on sourceforge projects and tell me that again. Number of bugs varies by project, not by open-ness.
The GeekNights podcast is going strong. Listen!
'The open-source implementation of TCP/IP in the Linux kernel clearly exhibits a higher code quality than commercial implementations in general-purpose operating systems,
Well of course it does! The Linux and BSD IP stacks are benchmarks. This is where practically all protocol research happens - how would anyone be able to verify your results otherwise? Furthermore, only the free stacks are useful for compatibility testing because they are so configurable.
So obviously it stands to reason that this code is much more complete and bug-free than any commercial implementation. THOUSANDS of people are studying every single line of this code on an ongoing basis.
I've worked on a number of commercial IP stacks - some from scratch, and some based on Linux. Any IP stack written from scratch is understandably simpler, but it's not that hard to implement the essential RFC requirements (i.e. the "MUST"s) and make it stable. Now, making it FAST and making it use all of the bleeding-edge TCP stuff... that's another story. Only Linux/BSD are there (and of course any other OSes which use their stacks).
The code for the shuttle's GPCs is closed, and it's regarded by many as probably the most bug-free code around with any degree of complexity. It's been upgraded several times since the '70s, and rarely have errors been found.
It probably had one of the longest development times for its size, too. Which helps a lot.
Quality has nothing to do with whether code is open or closed source. It's got everything to do with the environment in which it was written. Code written under extreme management pressure from a profit-hungry megacorp is just as bad as code written by an ignorant or uneducated dork in his basement.
Before we begin the bashing, let's note that two flavors of 2.4.19 were compared to two closed source Unix operating systems. Let's try to keep the evil empire out of this one!
Put identity in the browser.
TCP/IP is better on linux because many very talented people have worked on it. This is an area in which open source software development has worked well. However, it does not mean that open source developement always works better.
If they found 0.1 errors per 1000 lines of code, did they approach Linus and Co. to point them out? Has Reasoning submitted any kernel patches to address the errors they say they found?
Qu'on me donne six lignes écrites de la main du plus honnête homme, j'y trouverai de quoi le faire pendre.
Microsoft pinched their TCP/IP stack from *BSD
Not exactly true. I can't find the link off hand, but I read an explanation of the background to this myth quite recently. If you Google around you should be able to find it.
Back when MicroSoft were keen to add TCP/IP support to Windows, they contracted another firm to to do the work. That firm took the BSD licensed stack (from 4.3BSD as I recall), and did tyhe necessary porting work. This they then delivered to MS, meeting the original deadline. Since then, NT has gained a new TCP/IP stack written from scratch by MS engineers.
As a result, the TCP/IP stack currently used in Windows owes little or nothing to the BSD implementation.
Chris
The code I write for myself is the cleanest stuff in the universe. I get freaky about extra lines or lines that look "ugly" or inelegant.
.024 euros.
Now when I'm at work I toss out functional, ugly code. Doesn't work quite as well, but 90% of the users will never know that. I'll write catch statements for the most obvious errors, but I don't sit and brood about what some hypothetical idiot might want to do with the code. If there are enough people who hit an error there, I patch it, and move on with my life.
By and large, high production commercial code is sloppy. There isn't any profit to be made in making it pretty or elegant, and we all know how (for a random example) MICROSOFT feels about profit.
Open source is just the opposite; if you're not making any money on it, you're doing it for your own personal satisfaction, and I think most people find it more satisfying to have clean baddass code, rather than sloppy junk code. Heh. Especially when your NAME is on it, and the SOURCE is available.
Just my
ad logicam Claiming a proposition is false because it was presented as the conclusion of a fallacious argument.
Reasoning "sells automated software inspection services."
The key word here is "sells." They would have a tought time selling this to open sourcers, what with everything wanting to be free and all. Instead, they show the big closed source companies that their code isn't nearly as bug free as the open stuff, therefore they really need to buy this.
I'm not denying that open source is less buggy, but always question the motivation of the company making the claims. Just because Reasoning's assertions fit your own neat world views doesn't mean that they are without bias or secondary motivation.
Do not taunt Happy Fun Ball(TM)
Actually, you've inadvertantly stumbled upon an excellent point.
No code is perfect to begin with. The BSD stack is still improved from time to time. The BSD stack that companies folded into their code years ago has since had some major changes and the companies haven't bothered to take many of those changes into account.
Had they been required by license (GPL) to keep the code open, then it could be fixed by other people. Instead, the implementation has languished. This in fact is one of Stallman's great resons for keeping all code free.
However, the reality of it is that our current environment still favors closed source software. With any luck, people will slowly start to wake up and realize that source code needs to be open for all software projects. Think about it. If it was normal to receive source with binaries, nobody would really think twice about it. It's only seen as a bad thing because it's not what Microsoft does. But the reality is that Microsoft has a business model that works well for them, a giant monopoly. The reason their competitors fall on their asses is because they are trying to play as if they were MS, which they are not. It's not impossible to compete with Microsoft, it's just impossible to compete head-on.
What I am wondering is which is the cause and which is the effect:
Microsoft source code is defective because it is closed.
Microsoft source code is closed because it is defective.
now we need to go OSS in diesel cars
Now if they could please point out the filenames and line numbers in question, perhaps we could eliminate the bugs altogether...
Comment removed based on user account deletion
"Reasoning examined the TCP/IP stack and found fewer errors in Linux" The TCP/IP stack in Linux (and for that matter, most operating systems) was borrowed from BSD. Shouldn't this comparison be a testament to quality BSD instead of Linux? Paul Zimmerman http://zimmermantech.com/webcam.htm "Comments should be like skirts - Short enough to keep your attention, but long enough to cover the subject"
Listen to Live FM Radio
This is still an argument for the open source method, but I think that the code quality should be attributed to a different source. Perhaps it is not about an inherently good or inherently bad method. What if age is the key factor?
The Linux networking code has been in for a long time. Not in it's present form, obviously, but each change builds on the last; as it must in open source - it would be foolish to start afresh when you have something that works. So a cylcle develops and at each stage the code gets better. Compare this with proprietary; can they look at a competitors code? No. They must start afresh and so their code is effectively younger.
Further, if we measure software age not in units of time but in units of updates, open source has the advantage that there are many updates, there is always someone new to look at the code. No company can compete with the sheer quantity of viewings and therefore updates that occur in open source developments.
Carpe Daemon
"You mean it's been written with the latest design and coding ideas, to a high quality, tested, documentated..."
I have to respectfully disagree with you that this is a good thing. All too often students will learn a new design or coding idea and want to apply it even when it is not neccessary or the best tool for the job. Furthermore, students, in my experience, are way too ambitious to test much. The just want to code, code, and then code.
Finally, have you read much of the kernel? Documentation is sparse (though getting a little better in 2.5.x).
Office politics no. Dorm politics - e.g. my stack is better than yours? Maybe.
Holy s-, it's Jesus!
So why should any of this article be a suprise or even particulary note worthy?
perhaps because when large numbers of people are uneducated about something they use and make daily decisions about, it is shocking to them to learn that their assumptions (probably brought about by marketing) are erroneous.
Other notable, and obvious "surprises" in research:
-Two parents are better than one.
-More concealed carry = less violent crime.
-You are more likely to get sick at a hospital than at home.
-Breast milk is better for babies than formula.
A lot of money and time have been spent researching these topics, only to find what many of us already knew to be true and obvious.
Not everyone is educated and experienced in everything, and it can be painfully difficult to dissuade people of their delusions. Especially when they've been formed out of ignorance.
Two key points are that (1) most of the bugs Reasoning found are false alarms (which is an occupational hazard for this kind of analysis), and (2) one reason Linux does so well is that those lunatics at Stanford have been doing just this kind of analysis for quite some time, so most of the easily-found bugs were found long ago.
This doesn't invalidate any of their conclusions, of course: the Stanford lunatics haven't been analyzing NT, they've been analyzing Linux, and for sound academic reasons.
People coding something because they want to (and because they need it for something for themself) leads to better code. I know when I do something for myself, I don't half-ass it.
Coding for the end result = quality
Coding for a living = paycheck
Any questions?
In SOVIET RUSSIA... erm...NSA AMERICA, the Internet logs onto YOU!
The Linux IP stack is a complete rewrite and doesn't derive from the traditional BSD sockets code at all. In particular IP packet formation between Linux and BSD is completely different. The header and tail portion of an IP packet is handled in a single pass through called an "sk_buff". In BSD header and tail formation of the packet is handled in two passes, one for the header the next for the tail, in an "nbuf". The BSD protocol implementation is traditional and the one described in TCP/IP Illustrated, while the Linux implementation is completely new. I believe that one positive feature of the Linux implementation is that it has allowed for zero copy networking, though that's a limited benefit which is only of use to a very small subset of servers connected to very fast network links. A big positive of the BSD stack is that it's old, rigorously tested, and very well documented. Note that the System V Streams implementation is completely different as well, so Solaris and other SysV derived kernels follow their own method for packet formation. I make no claims that any of these protocol implementations are better than the others, only that the code base and history are completely different.
I've attended a few USENIX kernel internals courses but that's the extent of my competence (have poked through the source out of curiosity though). Please feel free to post additional information or correct any mistakes I may have made.
Cheers,
--Maynard
It is natural for open source projects that survive to become very high quality. Look at it this way: If you buy proprietary software from a corporation, you can be sure that they are motivated by the bottom line.
Corporations are there for one reason only: profit. This in itself does not mean that the products that they put out will be inferior. However, being motivated by profit means that:
1. They will push their employees to put out a product quickly.
2. If a product has flaws, it is the bottom line that dictates the priority given to fixing that flaw.
Open source on the other hand is completely different. Although it can be motivated by profit usually it is not as much. A lot of people do it because they just want to do it. This in itself does not make open source less buggy. I would say that most young projects have as many or more bugs in them than proprietary projects.
However, if the projects live for a long time it is because dedicated coders have decided to spend their time improving the product. This dedication over a period of time without the pressure by management to quickly push the product to market is the reason that open source becomes better than proprietary software.
The race isn't always to the swift... but that's the way to bet!