Thinking of Security Vulnerabilities As Defects
SecureThroughObscure writes "ZDNet Zero-Day blogger Nate McFeters has asked the question, 'Should vulnerabilities be treated as defects?' McFeters claims that if vulnerabilities were treated as product defects, companies would have an effective way of forcing developers and business units to focus on security issue. McFeters suggests providing bonuses for good developers, and taking away from bonuses for those that can't keep up. It's an interesting approach that if used, might force companies to take a stronger stance on security related issues."
If they weren't, they would be in the program design.
Thread over on the first post. Well done.
Of course they aren't defects, they should be treated as features!
We've treated potential vulnerabilities in our products, even extremely minor ones, as defects for over two decades now. And we have always given them very high priority.
To the best of our knowledge we've never had a remote exploit vulnerability, but even so we've gone so far as to scrap thousands of freshly pressed CDs a day before releasing them because I spotted a way to get root access through a tricky bit of business with shared libraries. (And that was for something spotted internally - no customer ever reported it.)
The real question isn't whether to treat security vulnerabilities as a defect - of course you do - but - somewhat paradoxically - whether or not to treat them as security vulnerabilities. We were acquired some time ago and have now adopted (and adapted to) various more complex procedures typical of a large company. There's this little box you're supposed to check in our current bug reporting system that says "this is a security vulnerability". The problem is that checking that box fires up a whole lot of extra process that rarely helps and can actually hinder prompt resolution of the problem and getting the fix into customer's hands.
If a user was intentionally mis-using software I had written, I wouldn't consider it a bug. Although a vulnerability is generally mis-use by someone other than the owner of that piece of software, I'd still have to conclude it's not a bug. If I'd built a car, I would be more than a little annoyed if I got the blame that someone had broken into it and run someone else over with it.
I think it needs to be left to the market to decide what is acceptably secure software. Many Ford cars from the early 90s had locks that were far too easy to break - just stick a screwdriver in and it opens - even did it myself when I locked the keys in the car once. They got a bad reputation, and Ford improved the security to a level the market was happier with.
The market in software doesn't work quite as well as for cars unfortunately, but that's another issue.
As a software developer I spend about a quarter of my time rewriting code that one of our other developers writes. His code is like a rhesus monkey came in and started flinging shit all around. He 'keeps up' with the other developers because he does the absolute minimum, never ever rewrites code to fix problems, cuts and pastes, etc. One time he cut and paste a second copy of a 200 line function so he could change one loop constant.
There's lots of developers like him, and they and/or their company should get sued over that code. At least when it is from negligence. Or there should be a licensing requirement.... something so that the people who are irresponsible or incompetent are held responsible for it.
Pretty much the only thing that makes programming not worth while is that people can hack out a 80% working code, get credit for it, then move on and leave all the crap for competent developers to fix. I would gladly pay a malpractice insurance fee if it means less having to deal with bullshit code.
Everybody, please laugh at the subject of my post which has no relation to its contents ;)
What I meant to write when I wrote the subject is that, from the point of view external to the organization developing insecure software, you are, according to the wisdom of the /. masses, supposed to vote with your wallet.
Yet, how's that expected to take place? To apply some of Schneier's observations, you have multiple parties, each with their own security agenda; the sysadmin might want the most secure option because anything less will be a nightmare to maintain, whereas the phb will want the cheapest because that'll make him look good in the eyes of those who set his salary.
Guess who makes the purchasing decision. Guess which security agenda will be reflected in that decision. Sometimes, the insecure option will be the cheapest even when the cost of bad security has been factored in.
Also, consider the fact that writing "perfectly secure code" is hard and time-consuming, and thus expensive. Given that it's hard enough to write reasonably non-buggy code when there's enough of us, what does that predict for security issues? Now add in the variability in skill level of the developers, and the varying experience with the particular code base they work on.
"The problem of course is I'm saying how the companies should handle them, and I have no authority at any of these places, save people actually valuing my ideas. Personally, I've done some development in the past, and there was the concept of defects. Your bonus would depend on how many defects were in your application at delivery time. These were feature-based defects, but shouldn't vulnerabilities be considered defects as well?"
So, the author freely admits he is neither a developer or a manager. If he was a developer he'd know that these are defects and everyone treats them as such.
If he was a manager, he'd know that one of the surest ways to wreck a good shop is to start doing comp based on defects. Here is what invariably (in my experience) happens when a shop includes defect counts in there comp plans.
1. Relationships between Dev, QA, Product Management and Operations get worse because the terms 'defect' and 'bug' become toxic. In reality these things always exist in software. The last thing you want to do is create barriers to dealing with them. Making the acknowledgment of a defect cost someone money means you will have arguments over every one of them unless they cause an out right crash.
2. Culture becomes overly risk-averse - No one wants to take on difficult problems or blaze new territory. The smartest people will naturally pick the easiest work to minimize the risk of defects.
3. Over-dependence on consultants - More CYA behavior. If it's too complex people will outsource to keep the defects away. This is a very bad thing if the nasty problems are because of business and not technical challenges. Now the people who know enough about the problem domain to understand the risk are hiring proxies who know nothing to avoid responsibility for 'defects'.
...the nature of the security issue.
A defect, by definition, is an unintended behavior of a program. Something was designed to work, but for whatever reason, doesn't. Compare this to a lack of a feature, which means that something doesn't work because there was never the intention for it to work in the first place.
A buffer overflow or SQL injection related issue is almost definitely a defect, since there is a dedicated, designed parsing mechanism to process input, and if some types of inputs are not processed as intended, it is a defect of the software.
On the other hand, for example, a security issue arising from plaintext transmission of sensitive data over the net, is not necessarily a defect. If the site in question was never designed to use SSL or another encryption mechanism, then it's a lack of a feature. If the site in question is an online banking site, then it is a blatantly poor and inexcusable design shortcoming, but nontheless, not a defect. (Of course, if the site DID intend SSL to work properly, but for whatever reason there is a hole allowing to crack or circumvent the encryption, then it IS a defect).
Besides, assigning a "defect" status to a security issue is not necessarily useful for it's own sake. The understanding is that a responsible company should treat a security issue with much higher priority than a non-security related one, defect or not (compare "we released an emergency hotfix to download" to a "we'll ship the patch in the next release cycle"). Saying a security issue is a defect, is like saying that a cardiac arrest is "organ numbness" - true, but not very useful.
When I think of defects and total quality management, I think of Edward Demings.
Edward Demings saw the problem of defects as a systems issue, not an individual performance issue. And his theory was that paying someone based on performance would have the unintended consequence of increasing the number of defects, not decrease them (Here is the list of Deming's 14 principles with my emphasis added in bold).
The article (at least in my reading) isn't saying that they should be held legally accountable as selling a defective product. Instead it's about how companies should approach a bug report of a vulnerability. He's saying, when someone reports a vulnerability, consider it something that you're obligated to fix, not as a feature request.
But then, I think most people do. It seems like he hit a bad support person.
I ran into a similar problem once with Citrix, actually. Their software was relying on some library that it assumed was installed, even though recent Linux releases (at the time) had stopped using that library. The result was that the software didn't work until you tracked down that library, dropped it in the right place, and then it worked fine.
So I went to their website to give feedback, just to let them know. I mean, I'm sure they would have figured it out, but I thought, "may as well give them a heads up" because it was happening on major linux distros almost a year after their release. Citrix had released several updates to their software, and never fixed this problem. I couldn't find anyplace on their website to provide feedback, except for a form to give feedback about the website itself.
So I wrote up a little feedback, trying to explain the situation briefly (i.e. "I wanted to drop some feedback to your development team letting them know there's a problem, how to fix it, but I can't find any contact information on your website. Is there any way to submit this sort of feedback). The response came back quickly, "If you want support, you'll have to pay for a support contract."
I wrote back again, trying to explain, "No, see, I'm not looking for help, I'm trying to be helpful. I'm letting you know that there's a problem I already know how to fix. I was just wondering if there was a place to submit this sort of feedback."
Again, the response came in, "I'm sorry sir, but if you want us to help you with this problem, you'll need to buy our support contract."
At that point, I gave up.
Also, if anything external to the way you work (i.e. the promise of more money) can make you work better, you're slacking off in your daily work: why don't you deliver peak performance without the extra money?
There's two ways to look at performance vs. compensation. Employees, ideally (at least from the employer's viewpoint) will look at it the way you do: you're being paid to do your best, so you should need no extra incentive to do so. Project management, on the other hand, should be pragmatic about it. Sure, employees SHOULD do their best no matter what, but maybe cash incentives can add motivation. If that is found to be the case, a good manager will choose results over principles.
In the RW, i'd suggset that we should consider the following;
You are Programmer Sian (notice the trendily androgynous name), you work for a gigantic software company, or conglomerate or industrial that does all its own major development inside, you are potentially confronted with;
1. Antiquated Developer Tools -- in general, the larger the development environment, unless you're Disgesting Your Own Pet's Nutrition, you are very likely to be using multi-year and/or multi-generation old development platforms and tools.
The question here is then, how can you effectively hold poor Sean accountable for vulnerablities, that are intrinsic to many older tools?
Who's more accounatable here? Sian or the managers who make the procurement decisions?
2. "Science Fiction" Application Programming Interfaces - depending on whether you are programming on a well-established product or not, if you are, Poor Sian is probably stuck with API's that were developed many years before and have been the victim of Design Creep, and its, Lunatic Cousin, Design Implosion.
In many instances the APIs, while they may once have had a large degree of Paradigmatic and Philosophic Design Integrity, as their initial Designers and Implementers have moved on to other; products, companies or, Worst Case, Inpatient Mental Health Facilities. Many New Designers have come in to add "Their Own Programming Uniqueness" to the APIs, frequently rendering the API's a jumble of radically different approaches to similar algorithms.
Should Sian be subjected to having their pay docked because 9/10 Functions implement a Library Call one way, and some "Johnny-Come-Lately" API function implements a similar looking, but substantially different in output function?
Shouldn't the API Designers/Architects be held more responsible for this one?
3. PHB Stupidity - As QC forwards endless application/OS defect notices to the Development/Maintenance Team, these defects are reviewed by the Team Managers and Supervisors. It's understandable, given the 11 hours per day of Absolutely Vital Meetings that most PHBs love to, i mean are forced to attend, that Defect Prioritization will suffer.
Sian can't choose what defects to repair, and in what order to repair them.
This is a management function, and one, in my experience, that Mgt usually jealously and zealously guards.
SOOOO, it's been the case in every Development project that i've worked on and know about, that PHB's have a well-understood tendency to prioritize Defect repair, according to external pressures, especially from Sales and Marketing.
Sales and Marketing organizations are usually likely to priortize according to their immediate impact on quarterly projections.
Vulnerablities are only likely to affect quarterly results when they are CATASTROPHIC defects, i.e. App or OS Killers. Otherwise, the majority of vulnerablities, which are usually well submerged in the Defect numbers, tend to get shoved aside for the higher priority defects that S&M believe impact immediate sales.
There are numerous other considerations here; including Contract Programmers, Legacy Compatability (ask the Vista Team about that one), Vendor Driver Teams that don't even know what to do with a new code base, etc, etc..
But it seems to me, that, while financial incentives CAN BE, useful as a Mgt tool for improving product quality, they should, to be even-handed, applied across the entire product team, with specific ***POSITIVE*** incentives used to take care limited, high priority problems across the product line.
There's already a tendency to "blame the programmer", and my Best Guess is, that any attempt to lay the responsiblity for vulnerabillites, THAT AREN'T CLEARLY THE RESULT OF SLOPPY/POOR/INCOMPETENT CODE PRODUCTION, at the feet of the programmer, will merely increase the employee turnover in the Development Team. Something that already is a problem most places.
from my experience: "The Fault, Horatio, Usually Lies Not In Our Code, But In Our Process"
Ten quid, she's so easy to blind. And not a word is spoken...
Microsoft don't seem to treat scurity vulnerabilities. Mind you, they don't seem to treat defects, either, so I guess they are still treated as the same.
It's a small world and it smells funny; I'd buy another if it wasn't for the money; Take back what I paid (SoM)
Actually you really need just one person in the company with "haxor" skills to test the security of the products that others make. A single person can very quickly find a lot of common holes. That person doesn't need to a developer. He/She can be there just for testing or even just for supervising others that make the testing, to make sure that they test for security vulnerabilities also.
The vast majority of security vulnerabilities are merely exploits of defects!
How do you hack a system? Find a bug, that's usually pretty easy....
Then you have the system operating already, "not as the designer intended" and you're more than halfway there...just add a bit of creativity and malice aforethought.
Functionality tests are easy to prove through unit and integration testing. Normal users spot functionality bugs quickly during normal product cycles.
However, security bugs are not easy to test or discover. In fact, it's very expensive to do testing to uncover even some easy classes of security vulnerabilities. Normal users do not stumble on security problems like they do with functionality issues.
Also, none of your developers were ever taught anything about application security in college. They professors are clueless. Even Michael Howard from MS who is hiring out of the best universities in the world cannot find a new grad who has any clue how to build secure software.
Functionality bugs and Security bugs are apples and oranges and deserve very different consideration. (Like measurement of Risk, etc)
Last, you can make a piece of software work. But you an never make a piece of software secure, only reduce risk to an acceptable level.
Horns are really just a broken halo.
I work with the vulnerability management team and product security team at a large software company, and trust me vulnerabilities are treated as product defects. The cost of addressing vulnerabilities in the field is huge, and not addressing them is simply not feasible - customers would never tolerate it.
The elephant in the room is that primitive, unsafe tools endlessly perpetuate these problems. Buffer over/under flows are not difficult problems to solve at language design level, but the common tools We currently use to create applications make diagnosing them and fixing them rocket science. C and C++ (and other lesser used languages) are notorious for being hostile to catching these problems at compile time or debugging them when they happen later. In most cases, the problem goes "unnoticed" affecting unrelated functions in the application downstream and incorrect behavior or crashes happen at a later time when they can no longer be traced back to the original cause.
For kicks check http://en.wikipedia.org/wiki/Buffer_overflow#Choice_of_programming_language
Google search on http://www.google.com/search?hl=en&q=%2Bbuffer+%2B%22overflow%7Coverrun%7Cunderrun%22&btnG=Search