Slashdot Mirror
With So Many Eyeballs, Is Open Source Security Better? (esecurityplanet.com)
Sean Michael Kerner, writing for eSecurity Planet: Back in 1999, Eric Raymond coined the term "Linus' Law," which stipulates that given enough eyeballs, all bugs are shallow. Linus' Law, named in honor of Linux creator Linus Torvalds, has for nearly two decades been used by some as a doctrine to explain why open source software should have better security. In recent years, open source projects and code have experienced multiple security issues, but does that mean Linus' Law isn't valid?
According to Dirk Hohndel, VP and Chief Open Source Officer at VMware, Linus' Law still works, but there are larger software development issues that impact both open source as well as closed source code that are of equal or greater importance. "I think that in every development model, security is always a challenge," Hohndel said. Hohndel said developers are typically motivated by innovation and figuring out how to make something work, and security isn't always the priority that it should be. "I think security is not something we should think of as an open source versus closed source concept, but as an industry," Hohndel said.
According to Dirk Hohndel, VP and Chief Open Source Officer at VMware, Linus' Law still works, but there are larger software development issues that impact both open source as well as closed source code that are of equal or greater importance. "I think that in every development model, security is always a challenge," Hohndel said. Hohndel said developers are typically motivated by innovation and figuring out how to make something work, and security isn't always the priority that it should be. "I think security is not something we should think of as an open source versus closed source concept, but as an industry," Hohndel said.
A: Other people
More is better when it comes to bugs that are mostly obvious to the typical person, but doesn't benefit complex code that whooshes over the head of 99.9% of people. My co-workers tell me I have an attention to detail. Some code will get 3-5 people looking at it and testing it over a period of a month or two, then the'll ask me to take a look. Many times I will find several bugs in 10-15 minutes by just reading the code, then I'll have questions about the code and ask them to run some tests that will further find some more bugs.
I do not trust normal humans to anything technical right. I would prefer languages that work better with static analysis, more free tools to provide quality static analysis, and more fuzz testing.
When software doesn't have visible source code, the legitimate users have no assurances regarding what it's doing, other than those imposed by the operating system (which they might not have complete source for either).
However, the bad guys still take the trouble to disassemble the code and find its vulnerabilities.
With many eyes, you still might not find all bugs, but you can, and can do so without the unreasonable investment of disassembling the code and reading disassembly - which is not like reading the real source code.
The larger issue is that we need publicly-disclosed source code for some things, to assure the public good, whether it is proprietary or Open Source. For example the emission control code in automobiles, which it turns out multiple manufacturers have defrauded.
Bruce Perens.
It depends. FOSS software often lacks QA, unit testing, code static/dynamic analysis and regression testing. Compared to a FOSS software with a similar QA process - I would say yes, more eyeballs make it better. Compared to a commercial software with a strict QA process - no.
Back in 1999, Eric Raymond coined the term "Linus' Law," which stipulates that given enough eyeballs, all bugs are shallow.
That's only true if those eyeballs are actually looking for bugs and organized enough to do something about them. Even then it's more like a principle than an actual truth. Some bugs are much harder to find than others no matter how many people are looking.
Linus is right... but note that he talked about eyeballs, not open vs closed source. If an open source project is obscure, or if the code is too hard to read, it may not get any scrutiny. On the other hand, closed source code from companies that care about security enough to pay security firms to scrutinize their code, or to hire security-knowledgeable developers and have them look at it carefully, can get a lot of eyeballs.
In the normal course of events, though, open source code almost always gets more attention than closed source, just because anyone who wants to look, can.
Note to ACs: I usually delete AC replies without reading them. If you want to talk to me, log in.
Open source has the potential for more eyes. It can be fixed after official support ends. It can be audited by anyone. Tampering with the code of open source may be more likely to be noticed. Tampering with closed source code can be more difficult. Commercial closed source developers may be more likely to be held accountable for vulnerabilities and thus can be more motivated to provide a secure solution. Commercial closed source may be more likely to be reviewed by a security expert because of there is not a security expert on the team, one can be hired, whereas for non commercial closed source, this may not be as likely an option. Closed source has security by obscurity, which can make some vulnerabilities somewhat harder to find by malicious actors.
My sig doesn't address Anons, sigs aren't visible to them.
While the "many eyes" can be theoretically a better model, practice has shown very few actually look at Open Source software with security in mind.
Even critically important projects like OpenSSL.
Security review takes time. Time is money (even in OSS world). Security audits require money. They don't get done, unless commercial entity (using OSS) commissions them.
The "many eyes" is a really bad security model in practice.
The fact is that there aren't many eyes on most parts of the code, and of the ones that are, very few of them are qualified to find the problems.
I use some software. It's in debian (and ubuntu) but hasn't been updated in years. It's not the sort of thing that needs to be updated. But I've manged to find and fix 6 or 7 bugs. serious bugs. coredump now type bugs. I've stopped reporting them back to debian because nobody is looking at bug reports. There is no upstream. I've reported them back to a fork I found on sourceforge but they asked me to rewrite my commit message without oxford commas. Seriously.
I'm glad it's open source, I wouldn't be able to use or fix it, otherwise. But nobody else will benefit from my bug fixes.
Copyright (c) 1990 - 2014 Dice. All rights reserved. Use of this comment is subject to certain Terms and Conditions.
See WordPress. Abysmal architecture, programmed by monkeys on crack, pretty good security. The last critical gap was closed after only 8000 websites had been infected, something like .0002% of the installbase or something.
Pretty neat.
I bet that gap in that obscenely expensive Oracle Java web application server thingie isn't found half as fast let alone fixed in such a speed.
We suffer more in our imagination than in reality. - Seneca
Any bug can be fixed after discovery because all eyes will be on it. (Or, at least, the eyes of most of the experts for that particular system.)
A huge part of security is being proactive throughout development---in design, code submission, review, and auditing.
Private companies can hire people to fill these roles as needed, but most open source projects rely on the security consciousness of individual contributors. Since security is often boring or counterproductive to the development of new features, I can easily see security being less of a priority for some developers.
Good security requires many eyes throughout the development process. It needs ongoing oversight to ensure that every module and code submission is consistent with the security model for the project.
BSD has one seriously security-conscious leader, but that is not typical. Maybe Red Hat will pay someone to oversee the security of the Linux kernel or audit its code, but most projects won't have that kind of backing. They'll rely on luck of the draw---maybe you attract someone with security expertise, or maybe you don't.
Without a dedicated security focus, projects should go through hardening phases where they deliberately welcome security experts and design/redesign as necessary for security. Even if it means a slowdown or moratorium on new features. Security takes time and effort, and the only solution is to put more eyes on it.
---
According to the latest ruleset, this post should be modded as Vorpal Flamebait +5.
Good point. You need assurance that the code in the device is the code that you see.
This is a way big problem for government. There are many integrated circuits in our fighter jets, etc. How do we assure that what is inside them is what we think? Thus, we have defense assurance programs that follow the production of a chip from design all of the way to the finished product. But these are a poor substitute for having a technical method to verify that the chip actually does what the manufacturer says it does, and nothing more. Nobody's figured out how to do that.
Bruce Perens.
Exactly. ESR summed up Linus's thoughts as ".. all bugs are shallow", not "all bugs don't exist".
Linus's exact words were:
"Somebody finds the problem, and somebody else *understands* it."
I'll share two examples from my own experience. Somebody found the shell shock bug and suggested a fix. Over the next few hours, hundreds of people looked at it. Some saw that the suggested fix wouldn't quite cover this variation or that variation, so they tweaked it. Florian Weimer, from Red Hat, said those tweaks would never cover all the variations, and suggested an entirely different fix, one that went to crux of the problem. Over the next few days, there was a lot of discussion. Eventually it became clear that Florian had been right. When he looked at the problem, he immediately understood it deeply. Well, it looked deep to us. To him, it was shallow.
""Somebody finds the problem, and somebody else *understands* it", Linus said. Stéphane Chazelas found shellshock, Florian understood it, fully, immediately.
There was no need to release a patch to fix the patch for the patch as we often see from Microsoft, or as we've seen from Intel lately. With hundreds of people looking at it, somebody saw the right solution, easily.
Here's another example from my personal experience with the Linux storage stack:
https://slashdot.org/comments....
When you claim it's better "because many eyes can look at it" the following questions are germane.
#1 - How many eyes ACTUALLY look at it, rather than just "have the opportunity"?
#2 - Of what quality are those eyes? Do they know the programming language in question? Are they up to date?
#3 - Is it just the code in question that matters? Or is it the code, plus a bunch of dependencies to other libraries, pre-compiled or otherwise? How long would it take to become not just a passing pair of eyes, but expert in the programming of the particular application?
#4 - Are those eyes likely to report what they find? Propose code solutions, or at least file bug reports?
#5 - Do the devs actually get around to acting on the proposed code or bug reports?
#6 - How fast do other people relying on this particular project stay up to date, either by downloading the code fixes and recompiling or downloading updated binaries?
It's not really any "safer" if it's open source but someone's pet project that gets an actual new version source/binary release every 6-12 months at most...
I have trouble with this industry-concept that software security should be put first -- it's an impossible business objective.
Think about how many industries focus on security. Banks, sure. Money transport, of course. Prisons and jails.
My air conditioner broke last week. It needed a new capacitor. It was a 5-minute $0 fix. Walk between the houses, open the compartment, pull out the breaker.
Now imagine your air conditioner, with the software industry's concept of security. Can you? How many check-points for a repairman to get to my air conditioner? How much added hardware? How much added expense in dollars and time? What stops someone from throwing a paint-filled balloon from fifty-feet away?
Security, when lives aren't at risk, is just so rarely worth it.
And when lives are at risk? Maybe you have a lock on your front door. Maybe it's a deadbolt. Maybe it's a really fancy locking mechanism, super-secure. Your front door is likely right next to a glass window. Congrats on the lock. Enjoy the bars on your windows.
And what stops your car, at highway speeds, from hitting another car at highway speeds? Right, a thin strip of white paint. Excellent. Sometimes the paint is yellow, even better.
We've never focused on security. We simply cannot afford to.
Instead, we talk about insurance, and criminal law enforcement.
So that's what I'm suggesting for software. Law enforcement. Deterrents.
Anything else, well, is just uncivilized.
My experience was, I saw a bug report in some open source and tried to fix it, and by the time I had a patch written a better one was already released upstream and I was the last person to upgrade because I was off trying to write a patch.
There are so many freakin' eyeballs available, volunteers are mostly just jerks like me who are getting in the way trying to help! You have to have an inside line to the developers or security researchers to even learn about a bug early enough to have anybody notice even if you understood it as soon as you heard about it.
Even writing new types of network servers; somebody announced they were abandoning a web middleware tool that was popular, and so I started plugging away at an apache module, but within a week somebody else released something similar enough to mine that I just stopped coding and used theirs. Sure, my architecture choices were better, but theirs weren't bad enough to amount to bugs so nobody would ever notice or care.
Programming is easy, the hard part is finding an unserved use case! And fixing known bugs is a pretty obvious use case.
I think there are a couple of aspects to this that might be a bit off the beaten track of threads posted so far...
The first is that we need to think about like-for-like comparisons. When these observations were initially made, 20 years ago, how many projects [either closed source or open source] were using automated source code scanning solutions? i.e. technology specifically written to parse code for flaws?
In other words, 20 years ago the "landscape" was likely to be close to "even". Today, however, many commercial software development shops use vulnerability scanning solutions and/or routinely conduct binary scans of resultant code. Today, many commercial development shops use automated test harnesses for load testing and regression testing. It is fantastic that they do. They do this because they can afford to and because the rapid advancement of this sort of technology has made it possible. Twenty years ago? Not so much.
This would suggest that we might start to see a difference in post-production bugs between Open Source and Commercial/Closed Source software where the development environments differ between these two operating models.
The second observation would be far more tenuous. In the same 20 year period, we have seen many different programming languages "come and go". Obviously the more established platforms (COBOL, C, C++, JAVA) continue to be popular, but this, too, brings differences in bug reports. The longer a language has been in existence, the more mature development becomes, the more libraries become available, the more skilled developers become in preventing even the more obscure bugs.
I don't have access to the data [and wouldn't know where to look for it, tbh] but I think it would be easy to graph out "average number of vulnerabilities per thousand lines of code" - i.e. defect density - over a 5, 10 or even 20-year period of language use. It would be reassuring to see if that trended down - but even more interesting [and worrying] if it didn't.
A while back I went looking to see if there were any "big rules" about different programming languages being more or less prone to vulnerabilities than others. I had read [maybe 25 years ago] that Ada was once thought of being a language with very few bugs. The theory was that it's compiler was so strict that if you could get your code to compile, it would probably run just fine. I was really surprised to learn that although there had been a few studies, there didn't seem to be any emergent evidence to suggest that there were differences between languages. I was surprised because my ignorance had suggested to me that helpful and/or heavily typed languages would be less bug-prone that more relaxed ones - i.e. that JAVA would have a lower defect density than C. Apparently [and I'd be happy for anyone to correct me] the evidence does not support this.
Sorry that this is trending away from the original question, but I think that context is absolutely crucial to get to a good answer to the original post - and that we would find that, like forecasting the weather, it would be pretty hard to do...
Your experiences remind me of something I learned about open source development. I now start by posting about what I intend to do. I've received these responses:
John is working on that and expects to release it next week.
No need to do all that, just use setting Xyx and skip the last part.
That seemed like a good idea, but when we looked into it we noticed this trap.
We decided we want Betaflight to focus on LOS. Your idea fits better with the iNav fork, which already does most of that.
Hey that's a good idea. Can you also allow multiples? That would be useful for me. I can help test.
It's pure BS. Yeah, you *can* look at the code, but how many do? And how many have the requisite knowledge to recognize it when something is wrong?
As noted on Slashdot over 10 years ago (https://it.slashdot.org/story/08/05/11/1339228/the-25-year-old-bsd-bug) it took 25 years to fix a bug in some commonly used open source. My understanding is that the Samba team even coded around the bug instead of looking at the code and getting it fixed.
Is open source security better than closed source? Sometimes yes; sometimes no. Depends on the developers, the projects and the companies involved. Security is about process and there's a lot more to the process than having access to the source code.
Somebody found the shell shock bug and suggested a fix. Over the next few hours, hundreds of people looked at it. Some saw that the suggested fix wouldn't quite cover this variation or that variation, so they tweaked it. Florian Weimer, from Red Hat, said those tweaks would never cover all the variations, and suggested an entirely different fix, one that went to crux of the problem.
Shellshock. A bug that was shown to have existed since 1989, and was patched in 2014.
I'm not sure this is the best example to use when discussing whether open source security is better than closed source security.
#DeleteChrome
So lets fix the original line, so 'given enough 'competing eyeballs all bugs are shallow'. You did not fail in your attempt to correct a bug, you played a role, you solution not as good as the other but still a comparison and next time, yours might be the better solution. Fixing bugs is about applying the best solution and having a range to choose from, whilst it does delay things, still works to ensuring the best solution at the time is used. So your effort was most definitely not wasted, just part of the overall competing eyeballs, looking for the best solution.
You might think you did a bad job but I think you did a good one, just part of the team effort. Sometimes you solution misses out and sometimes your solution will be the best one. Practice makes for better performance.
Chaos - everything, everywhere, everywhen