Slashdot Mirror
Thinking of Security Vulnerabilities As Defects
SecureThroughObscure writes "ZDNet Zero-Day blogger Nate McFeters has asked the question, 'Should vulnerabilities be treated as defects?' McFeters claims that if vulnerabilities were treated as product defects, companies would have an effective way of forcing developers and business units to focus on security issue. McFeters suggests providing bonuses for good developers, and taking away from bonuses for those that can't keep up. It's an interesting approach that if used, might force companies to take a stronger stance on security related issues."
If they weren't, they would be in the program design.
Thread over on the first post. Well done.
Of course they aren't defects, they should be treated as features!
Duh, that's like asking if potholes should be considered defects in a road.
you'll just have more disclaimers stating "this product does not provide secure services" which will continue to dominate the market because they are cheaper.
Unless you make security defects criminally punishable you will get no traction at all.
They'd get sued out of existence for shipping defective products. I can't see any company agreeing to label its products in such a way.
There are companies that DON'T treat security vulnerabilities as defects??
The last company I worked for (a fortune 100 company) considered them defects.
There was a concerted effort to track down and eliminate buffer overflows and other common security flaws.
Then again, after all the cutbacks (including the ones that cost me and my coworkers jobs) I'm sure it's not really a priority any more.
Unless you make security defects criminally punishable you will get no traction at all.
I'd imagine that this is already the case for banks, payment processors, medical facilities, and the like.
We've treated potential vulnerabilities in our products, even extremely minor ones, as defects for over two decades now. And we have always given them very high priority.
To the best of our knowledge we've never had a remote exploit vulnerability, but even so we've gone so far as to scrap thousands of freshly pressed CDs a day before releasing them because I spotted a way to get root access through a tricky bit of business with shared libraries. (And that was for something spotted internally - no customer ever reported it.)
The real question isn't whether to treat security vulnerabilities as a defect - of course you do - but - somewhat paradoxically - whether or not to treat them as security vulnerabilities. We were acquired some time ago and have now adopted (and adapted to) various more complex procedures typical of a large company. There's this little box you're supposed to check in our current bug reporting system that says "this is a security vulnerability". The problem is that checking that box fires up a whole lot of extra process that rarely helps and can actually hinder prompt resolution of the problem and getting the fix into customer's hands.
Here's a summary of the article:
Vendors should make their developers work more on security (via money)
Meh. How often are the developers free to choose which parts and aspects of their companies' software they want to work on? As long as the companies tell their developers what to work on, here's the easy way: tell the devs to work on security testing and fixing. Letting the developers manage themselves is not going to sit well with management types, so you almost always have developers who are told what to work on.
Also, if anything external to the way you work (i.e. the promise of more money) can make you work better, you're slacking off in your daily work: why don't you deliver peak performance without the extra money?
Yes, they are defects.
That said, there's one caveat to it: WHO'S defect is it? If the defect comes from a licensed library, OS issue, or other hidden cause, then that defect belongs to the source author/vendor.
Sometimes you end up having to work around someone else's crap.
If a user was intentionally mis-using software I had written, I wouldn't consider it a bug. Although a vulnerability is generally mis-use by someone other than the owner of that piece of software, I'd still have to conclude it's not a bug. If I'd built a car, I would be more than a little annoyed if I got the blame that someone had broken into it and run someone else over with it.
I think it needs to be left to the market to decide what is acceptably secure software. Many Ford cars from the early 90s had locks that were far too easy to break - just stick a screwdriver in and it opens - even did it myself when I locked the keys in the car once. They got a bad reputation, and Ford improved the security to a level the market was happier with.
The market in software doesn't work quite as well as for cars unfortunately, but that's another issue.
Bonuses based on code "quality" and or quantity has been tried before. It simply does not work. Counting the number of security issues some guy makes does not necessarily mean he or she is a good developer. It depends on several factors:
- difficult or simple code
- amount of code written
- where in the application does the developer code? Some code is more likely to result in security issues.
- how well that person knows the code
As a software developer I spend about a quarter of my time rewriting code that one of our other developers writes. His code is like a rhesus monkey came in and started flinging shit all around. He 'keeps up' with the other developers because he does the absolute minimum, never ever rewrites code to fix problems, cuts and pastes, etc. One time he cut and paste a second copy of a 200 line function so he could change one loop constant.
There's lots of developers like him, and they and/or their company should get sued over that code. At least when it is from negligence. Or there should be a licensing requirement.... something so that the people who are irresponsible or incompetent are held responsible for it.
Pretty much the only thing that makes programming not worth while is that people can hack out a 80% working code, get credit for it, then move on and leave all the crap for competent developers to fix. I would gladly pay a malpractice insurance fee if it means less having to deal with bullshit code.
No matter what tricks you try to use to get your developers and others to focus on security issues, it is going to cost money. Denying bonuses won't help because your developers can always leave and work for a competitor who doesn't play that game. And you'll still have to fix those vulnerabilities.
The solution is to ask your customers. Given the choice between a more secure, more expensive product and a less secure, less expensive product, which will your customers choose? Once you have the answer to that, you'll have the answer to whether you should think of security vulnerabilities as defects or a price-reduction feature.
To make software bullet proof requires the developers to have the skills of the hackers and malware writers, and the resources, the secret handshakes, the underground culture.
Yer talking one of two scenarios here:
Email me when that works out for ya~
Everybody, please laugh at the subject of my post which has no relation to its contents ;)
What I meant to write when I wrote the subject is that, from the point of view external to the organization developing insecure software, you are, according to the wisdom of the /. masses, supposed to vote with your wallet.
Yet, how's that expected to take place? To apply some of Schneier's observations, you have multiple parties, each with their own security agenda; the sysadmin might want the most secure option because anything less will be a nightmare to maintain, whereas the phb will want the cheapest because that'll make him look good in the eyes of those who set his salary.
Guess who makes the purchasing decision. Guess which security agenda will be reflected in that decision. Sometimes, the insecure option will be the cheapest even when the cost of bad security has been factored in.
Also, consider the fact that writing "perfectly secure code" is hard and time-consuming, and thus expensive. Given that it's hard enough to write reasonably non-buggy code when there's enough of us, what does that predict for security issues? Now add in the variability in skill level of the developers, and the varying experience with the particular code base they work on.
"The problem of course is I'm saying how the companies should handle them, and I have no authority at any of these places, save people actually valuing my ideas. Personally, I've done some development in the past, and there was the concept of defects. Your bonus would depend on how many defects were in your application at delivery time. These were feature-based defects, but shouldn't vulnerabilities be considered defects as well?"
So, the author freely admits he is neither a developer or a manager. If he was a developer he'd know that these are defects and everyone treats them as such.
If he was a manager, he'd know that one of the surest ways to wreck a good shop is to start doing comp based on defects. Here is what invariably (in my experience) happens when a shop includes defect counts in there comp plans.
1. Relationships between Dev, QA, Product Management and Operations get worse because the terms 'defect' and 'bug' become toxic. In reality these things always exist in software. The last thing you want to do is create barriers to dealing with them. Making the acknowledgment of a defect cost someone money means you will have arguments over every one of them unless they cause an out right crash.
2. Culture becomes overly risk-averse - No one wants to take on difficult problems or blaze new territory. The smartest people will naturally pick the easiest work to minimize the risk of defects.
3. Over-dependence on consultants - More CYA behavior. If it's too complex people will outsource to keep the defects away. This is a very bad thing if the nasty problems are because of business and not technical challenges. Now the people who know enough about the problem domain to understand the risk are hiring proxies who know nothing to avoid responsibility for 'defects'.
...the nature of the security issue.
A defect, by definition, is an unintended behavior of a program. Something was designed to work, but for whatever reason, doesn't. Compare this to a lack of a feature, which means that something doesn't work because there was never the intention for it to work in the first place.
A buffer overflow or SQL injection related issue is almost definitely a defect, since there is a dedicated, designed parsing mechanism to process input, and if some types of inputs are not processed as intended, it is a defect of the software.
On the other hand, for example, a security issue arising from plaintext transmission of sensitive data over the net, is not necessarily a defect. If the site in question was never designed to use SSL or another encryption mechanism, then it's a lack of a feature. If the site in question is an online banking site, then it is a blatantly poor and inexcusable design shortcoming, but nontheless, not a defect. (Of course, if the site DID intend SSL to work properly, but for whatever reason there is a hole allowing to crack or circumvent the encryption, then it IS a defect).
Besides, assigning a "defect" status to a security issue is not necessarily useful for it's own sake. The understanding is that a responsible company should treat a security issue with much higher priority than a non-security related one, defect or not (compare "we released an emergency hotfix to download" to a "we'll ship the patch in the next release cycle"). Saying a security issue is a defect, is like saying that a cardiac arrest is "organ numbness" - true, but not very useful.
The EULA's exempt holding the software maker for defects in their products anyway. Be them security holes or total meltdowns.
---- Booth was a patriot ----
When I think of defects and total quality management, I think of Edward Demings.
Edward Demings saw the problem of defects as a systems issue, not an individual performance issue. And his theory was that paying someone based on performance would have the unintended consequence of increasing the number of defects, not decrease them (Here is the list of Deming's 14 principles with my emphasis added in bold).
The article (at least in my reading) isn't saying that they should be held legally accountable as selling a defective product. Instead it's about how companies should approach a bug report of a vulnerability. He's saying, when someone reports a vulnerability, consider it something that you're obligated to fix, not as a feature request.
But then, I think most people do. It seems like he hit a bad support person.
I ran into a similar problem once with Citrix, actually. Their software was relying on some library that it assumed was installed, even though recent Linux releases (at the time) had stopped using that library. The result was that the software didn't work until you tracked down that library, dropped it in the right place, and then it worked fine.
So I went to their website to give feedback, just to let them know. I mean, I'm sure they would have figured it out, but I thought, "may as well give them a heads up" because it was happening on major linux distros almost a year after their release. Citrix had released several updates to their software, and never fixed this problem. I couldn't find anyplace on their website to provide feedback, except for a form to give feedback about the website itself.
So I wrote up a little feedback, trying to explain the situation briefly (i.e. "I wanted to drop some feedback to your development team letting them know there's a problem, how to fix it, but I can't find any contact information on your website. Is there any way to submit this sort of feedback). The response came back quickly, "If you want support, you'll have to pay for a support contract."
I wrote back again, trying to explain, "No, see, I'm not looking for help, I'm trying to be helpful. I'm letting you know that there's a problem I already know how to fix. I was just wondering if there was a place to submit this sort of feedback."
Again, the response came in, "I'm sorry sir, but if you want us to help you with this problem, you'll need to buy our support contract."
At that point, I gave up.
Security Tracker, best source of information on security vulnerabilities that I know of.
companies would have an effective way of forcing developers and business units to focus on security issue
I don't think developers need to be 'forced' because generally they understand the importance of making their software secure and if they don't do it then it's usually due to external pressures such as unreasonable deadlines and management not wanting them to spend time on something that does not have tangible results.
In short, the problem with a lot of companies is that management doesn't value security as much as it should. If they did then they probably would handle vulnerabilities as defects - but then again a lot of these vulnerabilities would have been prevented to begin with by giving developers sufficient time to properly design and test their code.
These legal concepts apply to cars and other tangible goods but not to software. They should.
Would that not destroy hobby software, and much of OSS and Free Software along with it?
You are correct. A security vulnerability is a defect if and only if the defect represents a failure with respect to a requirement. In fact, security is but one dimension of reliability, and so if the vendor is responsible for reliability, then the vendor should also be responsible for security. If there is no requirement for security, then it is not a defect. One of the posters mentioned that software that is sold for the intended purpose of general use on the Internet carries with it some implied requirements. (Merchantability.) Thus, just because the requirement for security is not explicitly stated does not mean that it is not there. If security is a requirement for a given environment of use, then a vulnerability is a defect, by definition. And yes, producers of software should be accountable in some manner for any and all defects, according to the terms of any explicit or implied warrantee.
The people who produce and sell software already thought of this. The EULA goes to great lengths to try to absolve the maker from any kind of liability. In fact the data sheets for chips have wording that says the chips are not for use in any application where human life may be at risk.
If I were seriously damaged because of defective software or hardware, I would expect my lawyer to argue that the manufacturer had at least some liability. Microsoft, in particular, has very deep pockets though and I wouldn't be willing to spend much money fighting them. What I would hope for is that the threat of bad publicity would cause them to give me some money to shut up.
Anyway, as far as treating exploits as product defects, ... good luck.
"McFeters suggests providing bonuses for good developers, and taking away from bonuses for those that can't keep up. It's an interesting approach that if used, might force companies to take a stronger stance on security related issues."
They already are considered defects. The only thing that might change is the prioritization of defects. This topic is ridiculous.
Bonuses? Are you suggesting bonuses for finding defects? In 20 years I've worked at only once place that awarded bonuses for resolving defects. The idea failed within weeks. I thought everyone had come to realize it was a bad idea and for obvious reasons. Must be a new generation of devs taking to the reigns and retrying old dead ideas.
You know what force[s] companies to take a stronger stance on security related issues? The consumer. Assuming the consumer knows anything and isn't just buying the same thing his cousin did because his cousin knows a guy who knows a guy who is a half-wit "computer consultant". Too many community college consultants out there...
In the RW, i'd suggset that we should consider the following;
You are Programmer Sian (notice the trendily androgynous name), you work for a gigantic software company, or conglomerate or industrial that does all its own major development inside, you are potentially confronted with;
1. Antiquated Developer Tools -- in general, the larger the development environment, unless you're Disgesting Your Own Pet's Nutrition, you are very likely to be using multi-year and/or multi-generation old development platforms and tools.
The question here is then, how can you effectively hold poor Sean accountable for vulnerablities, that are intrinsic to many older tools?
Who's more accounatable here? Sian or the managers who make the procurement decisions?
2. "Science Fiction" Application Programming Interfaces - depending on whether you are programming on a well-established product or not, if you are, Poor Sian is probably stuck with API's that were developed many years before and have been the victim of Design Creep, and its, Lunatic Cousin, Design Implosion.
In many instances the APIs, while they may once have had a large degree of Paradigmatic and Philosophic Design Integrity, as their initial Designers and Implementers have moved on to other; products, companies or, Worst Case, Inpatient Mental Health Facilities. Many New Designers have come in to add "Their Own Programming Uniqueness" to the APIs, frequently rendering the API's a jumble of radically different approaches to similar algorithms.
Should Sian be subjected to having their pay docked because 9/10 Functions implement a Library Call one way, and some "Johnny-Come-Lately" API function implements a similar looking, but substantially different in output function?
Shouldn't the API Designers/Architects be held more responsible for this one?
3. PHB Stupidity - As QC forwards endless application/OS defect notices to the Development/Maintenance Team, these defects are reviewed by the Team Managers and Supervisors. It's understandable, given the 11 hours per day of Absolutely Vital Meetings that most PHBs love to, i mean are forced to attend, that Defect Prioritization will suffer.
Sian can't choose what defects to repair, and in what order to repair them.
This is a management function, and one, in my experience, that Mgt usually jealously and zealously guards.
SOOOO, it's been the case in every Development project that i've worked on and know about, that PHB's have a well-understood tendency to prioritize Defect repair, according to external pressures, especially from Sales and Marketing.
Sales and Marketing organizations are usually likely to priortize according to their immediate impact on quarterly projections.
Vulnerablities are only likely to affect quarterly results when they are CATASTROPHIC defects, i.e. App or OS Killers. Otherwise, the majority of vulnerablities, which are usually well submerged in the Defect numbers, tend to get shoved aside for the higher priority defects that S&M believe impact immediate sales.
There are numerous other considerations here; including Contract Programmers, Legacy Compatability (ask the Vista Team about that one), Vendor Driver Teams that don't even know what to do with a new code base, etc, etc..
But it seems to me, that, while financial incentives CAN BE, useful as a Mgt tool for improving product quality, they should, to be even-handed, applied across the entire product team, with specific ***POSITIVE*** incentives used to take care limited, high priority problems across the product line.
There's already a tendency to "blame the programmer", and my Best Guess is, that any attempt to lay the responsiblity for vulnerabillites, THAT AREN'T CLEARLY THE RESULT OF SLOPPY/POOR/INCOMPETENT CODE PRODUCTION, at the feet of the programmer, will merely increase the employee turnover in the Development Team. Something that already is a problem most places.
from my experience: "The Fault, Horatio, Usually Lies Not In Our Code, But In Our Process"
Ten quid, she's so easy to blind. And not a word is spoken...
This reminded me of a funny/ typical/ stupid/ aggravating thing at work a few weeks ago. I pointed out a security vulnerability in one of our intranet apps during a meeting to discuss the next release. Despite exasperating efforts to educate --and a heated argument over the correct term-- a project manager insisted on spreading the word to upper management that we had a "security breach." But in the war with management (and those who THINK they're above us on the org. chart), I guess it's all about the power struggle.
If you don't work with idiots, count yourself lucky.
Ask me about my sig!
The vast majority of security vulnerabilities are merely exploits of defects!
How do you hack a system? Find a bug, that's usually pretty easy....
Then you have the system operating already, "not as the designer intended" and you're more than halfway there...just add a bit of creativity and malice aforethought.
The problem of security holes in commercial software products is not one of developer apathy, but instead is a consequence of resource constraint. Which is just a nice way of saying that during the push to achieve an unreasonably accelerated product launch date with a short staff, small things get overlooked by developers, and the big things get overlooked by management.
"Hey, boss, we've got a potential remote exploit here. We can't ship this garbage." "We have to ship. We'll catch it on the first patch cycle." "Uh, boss, we've never before caught anything on the first patch cycle. Why should we expect this one be any different?" "Good question. Here's another: Who's going to sign your paycheck next week if we don't ship this product on time?"
Clueless writer dorks should know when to shut up.
Warning: This signature may offend some viewers.
Management are trying to maximise profit, and typically don't care anywhere near as much as developers whether the job is done 'right'.
The problem is most buyers of software are way more interested in shiny bits and pieces than the security. If (more) people weren't willing to put up with insecure software, managers would be asking the developers to work more on the security aspects of the application.
Functionality tests are easy to prove through unit and integration testing. Normal users spot functionality bugs quickly during normal product cycles.
However, security bugs are not easy to test or discover. In fact, it's very expensive to do testing to uncover even some easy classes of security vulnerabilities. Normal users do not stumble on security problems like they do with functionality issues.
Also, none of your developers were ever taught anything about application security in college. They professors are clueless. Even Michael Howard from MS who is hiring out of the best universities in the world cannot find a new grad who has any clue how to build secure software.
Functionality bugs and Security bugs are apples and oranges and deserve very different consideration. (Like measurement of Risk, etc)
Last, you can make a piece of software work. But you an never make a piece of software secure, only reduce risk to an acceptable level.
Horns are really just a broken halo.
It a remote attacker is able to exploit a buffer overflow to run arbitrary code via a network service (that's intended to be publicly reachable), then that's a software code defect.
The defect is the buffer overflow that can occur -- the security vulnerability is that the bug can be used by a remote attacker to cause the program to do something that compromises security.
Security vulnerability effects the severity of the bug, but not the fact that it is a bug.
If the vulnerable network service is documented as to be run only behind a special firewall, insulated from any adversary, then it's not a security vulnerability, if it is "exploited", then the exploit was due to the configuration of the software and network (for which the network admin is responsible). There is still a defect in the software code (but independent of security issues that improper system configuration may create).
If the program by default allows commands to be remotely executed through the software, in a normal way, that is part of the product, (without exploiting a bug), then it's not a defect.
Although it may have security implications, especially if the feature is enabled by default and there are no security protections by default.
If the software (that allows remote command execution) has no possibility of implementing security over the feature, and it is intended to provide a network service, then it is a design defect (not a programming defect).
If the software can be secure, but the default is not, then this is no defect -- either the documentation explains how to secure it, and the user is in error, or it doesn't, and the documentation is defective.
Insecure defaults aren't desirable, but they're not defects. The insecure defaults may be very convenient when running the software in a recommended environment where security is not an issue, or security is provided by network equipment (configured according to recommendations of the documentation to disallow untrusted access, say to certain ports, or from untrusted IPs).
I work with the vulnerability management team and product security team at a large software company, and trust me vulnerabilities are treated as product defects. The cost of addressing vulnerabilities in the field is huge, and not addressing them is simply not feasible - customers would never tolerate it.
Redefining something rarely changes anyone's behavior.
Vunerabilities aren't defects unless the behavior of the application is inconsistent with it's requirements.
In any case, the solution isn't likely to be found in rewarding or punishing developers, but rather making security part of the requirements and providing enough development and testing time to insure that the software is secure.
Generally the market drives the process rather than software quality.
How many even slightly large software products have you ever seen that have no outstanding defects to begin with? Just browse a buganizer or two. There are thousands of open issues for major apps; some false, but plenty viable, if often minor, "defects." How you refer to a coding or logic error is irrelevant; whether it gets fixed or not is entirely dependant on the dedication of the supporting company, and how they envision it affecting their bottom line.
On the other hand, for example, a security issue arising from plaintext transmission of sensitive data over the net, is not necessarily a defect. If the site in question was never designed to use SSL or another encryption mechanism, then it's a lack of a feature.
I disagree. What I would conclude in this case is that it is not the fault of the coder. It is the fault of the company that provided the software though.
Such a product would be designed in many stages. You would have an analysis, a design and an implementation phase, where coding is done (possibly in small iterative steps as in extreme programming, but all the steps are there). In this case the analysis, design or both are faulty. This is not the poor coders fault, but the coder was given defective specs, so the defect is there. The coder implemented correctly a defective system.
If the site in question is an online banking site, then it is a blatantly poor and inexcusable design shortcoming, but nontheless, not a defect. (Of course, if the site DID intend SSL to work properly, but for whatever reason there is a hole allowing to crack or circumvent the encryption, then it IS a defect).
Try telling the bank if it loses millions as a result that this was not a defect. This is a sure way for your company to lose all credibility.
Poor Micro$oft would go belly up!
That's too broad of a question; it depends on the vulnerability. If my car's door lock can be bypassed by pulling slightly sideways on the door handle, that's a defect. If I can hold a cell phone over my car's windshield and it triggers the remote door unlock, that's a defect. But if I can smash a car window and climb in through it that's not a defect. If I can use an advanced lock pick set and have 2 hours alone with a car and get in that's not a defect.
The same thing applies in software - no piece of software will ever be impenetrable to every attack. The question is a matter of balance - given the purpose of the software, how much security is expected or required? How much trouble does a malicious entity have to go through to compromise the software?
And for many people the answer to those questions will be different for the same piece of software. That's where layered security comes into place. If I'm worried about my car's window being smashed and somebody climbing in to it, I don't blame the car manufacturer, instead I park the car in a garage. If there is something extra valuable I want to put in the car in the garage, maybe I put it in a lock box.
Well of course vulnerabilities should be considered defects. Even more so, DRM should be seen as defects as that is even in the program's planning. A security hole is a small cut, something that should be patched, DRM is like a gaping wound that will never be patched.
Taxation is legalized theft, no more, no less.
Developers who release shoddy products should be treated as engineers who design flawed systems. Unlike engineers, however, the quality of software developers has started to dwindle. What with outsourcing and economic uncertainty ...
Then again, you have to consider that vulnerable programs don't often contain vulnerable code themselves. Some link to vulnerable libraries and so forth. Some vulnerabilities are just so obscure that developers can't anticipate them.
Well then, the moral of your story is: don't buy/use Citrix' products. (I feel for you and I've the done the same when such has occurred to me --i.e., try to help out-- but the most of the proprietary world doesn't doesn't care.)
Very true. Also, the bonus system won't work for two reasons:
1: Security bugs tend to be discovered years down the road.
2: A lot of companies would have to start paying bonuses to programmers, and not just to S&M. That'd never float.
"when someone reports a vulnerability, consider it something that you're obligated to fix, not as a feature request."
Why any company in the world would do something like that!!!???
Oh, yes, only if they are legally or financially forced, that's how.
Or do you think any company in the world would rise their production costs for no benefit?
The elephant in the room is that primitive, unsafe tools endlessly perpetuate these problems. Buffer over/under flows are not difficult problems to solve at language design level, but the common tools We currently use to create applications make diagnosing them and fixing them rocket science. C and C++ (and other lesser used languages) are notorious for being hostile to catching these problems at compile time or debugging them when they happen later. In most cases, the problem goes "unnoticed" affecting unrelated functions in the application downstream and incorrect behavior or crashes happen at a later time when they can no longer be traced back to the original cause.
For kicks check http://en.wikipedia.org/wiki/Buffer_overflow#Choice_of_programming_language
Google search on http://www.google.com/search?hl=en&q=%2Bbuffer+%2B%22overflow%7Coverrun%7Cunderrun%22&btnG=Search
...the source of the crapacitors for fraud.
Ideally, yes all program bugs, and security holes should be treated as defects.
In reality, programs can run to millions of lines of code worked on by hundreds or even thousands of different people.
On a scale that large, no amount of proof reading an double checking will work, there is just too much there. Its why we see beta tests, its an attempt to put their program under real world conditions to see what breaks.
If they weren't, they would be in the program design.
"It's not a bug; it's a feature."
That old joke isn't always a joke. Some vulnerabilities are built in, because they were put there intentionally by the designers and/or developers.
This is one of the primary arguments behind Open Source: If you can't get at the code and study it, you don't have any idea what "special features" might be hidden in there. The people who built it could have provided all sorts of back doors for exploitation by themselves or by other customers who have paid them for knowledge of how to get into your system.
Needless to say (but I'll say it anyway), one of the names for such non-bug vulnerabilities is "DRM", which refers to hidden code that prevents you from using your system as you'd like to use it.
Consider also the recent stories about software in Vista and earlier Windows releases that does automatic updates to some system components, even when you think you've turned off automatic updates. This misfeature is there explicitly to allow Microsoft (or anyone who pays them for access) to replace parts of your system with new code. It's pretty clear that this is a "vulnerability", but it's not a "bug" or "defect", because it was intentionally designed into the product.
Those who do study history are doomed to stand helplessly by while everyone else repeats it.
They may be communicated to the public in a different way, but from a development perspective they're defects. The article's author describes a software vendor treating vulnerabilities as feature requests. This reminds me of Apple's response to the recent Safari "carpet bomb" controversy. Any possible vulnerability is a defect, usually a high priority one IMO.
What about customer satisfaction, and the financial detriment of losing your customers?
Dev1: Yeah, don't check for buffer 0o0verflow ...f'D up there asses...f'D up there asses...
Dev2: So when we quit/getFired()...they can never check all that code
Dev1:
for (i=0; iinfin;i++)
{
attackIE()
coverTracks(myIP)
closeSocket()
}
It's not so much a legal question as a business problem. Vendors are obligated to fix their bug and security holes because they are problems for customers.
I would just leave the courts out of it and let the free market decide. In this particular case, a good response would be: (A) I have reported this to management and we will seek other vendors to suit our needs. (B) I am obligated to report this security hole to the usual vulnerability lists, because since I found it obviously somebody else could too.
If they still don't fix it after that then find another vendor. Eventually, *poof* - bad vendor goes out of business, problem solved.
Security vulnerabilities are a backwards way of describing the issue. There are "security issues" and there are "vulnerabilities". If you throw user passwords into the cloud, it's a security issue. If there's a way for someone to hack into your database of passwords, it's a vulnerability.
Vulnerabilities don't get solved -- virtually ever. Vulterabilities get "handled". By that I mean that every "solution" costs something -- resources, performance, interactivity, flexibility, time, something.
For example, the vast majority of the systems I build for customers do not encrypt user passwords, for one very simple reason. When a customer calls and says that they've forgotten their passcode, "resetting it" isn't as good customer service as simply telling them what it is. It's purely a business issue -- the added security of encrypting their passcode is simply not worth guiding them through the resetting process, or changing it and leaving them to wonder what it was. Simply put, they called to ask what their password is. They expect to receive that piece of information. We give it to them. Outside of systems dealing with actual dollars, the security benefit is not important enough.
I always look to the city streets. A typical 6-lane road (well they're typical where I live), has an orange line, four dashed white lines, and two solid white lines. Cars, buses, bicycles, motorcycles, and pedestrians travel in two directions at varying speeds. Absolutely nothing stops me from flicking my wrist to the left and swerving into on-coming traffic causing a huge and likely fatal problem to dozens of people going 80km/h.
If my city streets were a typical software application, would you have me ensure that motorists couldn't hit each other? Rules aren't enough, obviously. Walls and baracades would replace painted lines. You couldn't change lanes willy-nilly. And hey, every traffic light would need to raise tire-spikes? The costs, maintenance, and general complexity of everything and everyone on the roads just boils over.
The roads are built for travelling. As long as the primary focus is travel, it works. Everything else can be on the side-lines, but they cannot become the main event. Yes the roads could be safer, but no it's not worth losing the phone, the radio, the passenger conversation, the freedom, the flexibility, the speed, and the choice that we currently have. You know, it would be a lot safer if everyone drove minivans. And it would be a lot cheaper if everyone rode bicycles. And it would be alot faster if everyone rode motorcycles. But "everyone" and "all the time" just isn't an option in a free society.
The same is true with software. If I can send an e-mail as easily as opening a socket, connecting with my destination, issuing three commands, and transmitting my message, that's a beautiful thing. But now if you make me follow six rules for how and when to do so, lookup eight registries of my destination, dodge content restrictions, authenticate myself, wait for them to investigate me, respond to the investigation requests, respond to challenges, scan my own out-going message, and have them scan the very same message on the other end, all to deliver the very same tiny message, then we've taken a 1K communication and made it a 30K transaction.
Case in point. I do a lot of e-commerce work with a lot of gateways. Some -- the old-fashioned ones -- accept a very simple url-encoded post of name value pairs. Something as simple as an account number, a credit card number, and an amount to debit, they return an approval code. It can be that simple -- 40 bytes sent, 10 bytes received. Others, woah others, demand 3K of xml code to describe dozens of logically optionally yet syntactically mandatory fields, and return 2K of re-iterated information along with the same single approval code.
Sure, when I'm doing something incredibly fancy, the small one becomes just as big as the big one. But 99.9999999% of the time, I just want to charge a credit card. And I
I agree totally.
Also, the customers aren't prepared to cough up what it takes to make the applications secure.
Some say this is a myth. I'd say that is not.
I have only met a few that even ask questions about application security.
We always try to sell security measures as a part of the solution. It is nearly impossible to make the customers understand without sounding paranoid and losing credibility.
And yes, I work a lot with banks and other similar entities which are supposed to have really high levels of security.
In some areas(that would be those where the government agencies are breathing down their necks), they are really good and and have enormously bureaucratic(verging on PIA) ways to handle that security.
But in almost ALL other areas of their application jungle, they aren't the least better, or even worse, than Joe's accounting down at the corner.
To end on a positive note, I did read a tender invite(i like it that way) for a new thing just the other day that explicitly excluded "office-like" products from being used in the solution.
Not that we neither would, or even could do that anyway, but maybe they are getting it, after all.
However, this is the first time i have ever seen this.
Baboons are cute.
There is no bug free code. You are dependent on SO many variables, most of them not under your control (your program runs on an OS that you didn't write, was compiled with a compiler you didn't write, runs on hardware you didn't design which uses a BIOS that's not under your control...), that you can never credibly claim your program runs flawlessly. Even any given Hello World has the potential to be buggy, not even due to you. Worse yet, any (but the compiler, I give you that) can change on the target machine at any point in time. And any change may make a flaw come out that you couldn't even test when you implemented the software because the machine that uses a certain flag/spec you didn't account for when you created the software, simply because it wasn't even on the horizon back then.
There are two possible ways commercial software can be done should such a thing be implemented: First of all, not at all. It's an insane risk to be possible responsible for a flaw that's not going to show until the next generation of hardware or OS. Or software becomes insanely expensive due to increased testing and companies using the additional revenue to "insure" themselves against the risk of future lawsuits.
I do agree that the current practice of producing bananaware and abusing the customer as a paying beta tester is despicable, but simply turning around by 180 degree isn't going to solve the problem. There is no black and white solution to this problem.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
"What about customer satisfaction"
bollocks
"and the financial detriment of losing your customers"
Which part of "legally or financially forced" didn't you understand?
Wow. You'd never expect a significant software company to do anything like that.
Wait... That process is *exactly* the Microsoft process.
meh
Um... you know the two are connected, right?
seriously! is there any company that treats vulnerabilities as features as opposed to defects?
Vulnerabilities shouldn't be treated as defects, they _are_ defects. Defects of the most critical level. Which possibilities do you have to detect tampered data?
cb
I don't know if this is still the case, but for several years in this century the only defect reporting system for IBM Lotus Notes was to describe the problem in a physical paper letter and mail it to a P.O. Box. I couldn't make that shit up.
There are several problems with the "unknown" factors in software design and result. One cannot fully determine what all features should or should not be in a product PRIOR to its design and completion. I think there is a law about software design that controls this. ;)
However, if the final product deviates from the design spec, i.e. if there are "features" (positive or negative) that end up in the product, then they must be addressed as "feature" or "bug". In either case, the design spec must be visited and altered to include or exclude a specific "feature".
For instance...
Both Space Shuttle disasters happened because the designers - testers - and maintainers all failed to evaluate the "feature" against the specification and make a ruling on if it was a flaw or not.
The bulging rocket seal and the extrusion that occurred in the first disaster...they said, "well it never blew up before..." and that was their excuse for not fixing it.
Or the foam that fell off in the second disaster, damaging the wing of the shuttle..."it never did any damage before..."
Famous last words...
This is why Scrum is such a great framework in which to develop software (or hardware)...you develop the item, and test it along the way...in testing you verify the functionality of the features against the spec. If you find things don't work, you address them in the next sprint (you don't change the spec to meet the new functionality...) if you find things not defined in the spec, you address them in the next sprint and decide if the functionality is beneficial or detrimental to the over-all functionality.
This would have resolved the issues with the shuttles, and with much of Microsoft's software problems...
So as we look to Web 2.0 and beyond, we should evaluate in what framework we operate...so far, the one that is most common is the one giving us the most trouble...
Check out scrum, Ken Schwaber's book is very good and can be applied to many situations.
Obviously, I haven't given this topic the same amount of thought that the people who would argue that vulnerabilities aren't defects...
It would seem to me that reaching such a conclusion would involve some pretty thorough and creative thinking, possibly involving the use of alcohol, narcotics or some sort of newspeak habit, or, of course, possible combinations of such...
I'm not sure that I can completely rule out sheer stupidity either, but that would also have gone without saying.
Oh, well, didn't someone once say that reality beats imagination on a daily basis? I certainly never imagined that level of tomfoolery...
F
"The number you have dialed is imaginary. Please rotate your phone 90 degrees and try again."
We've treated potential vulnerabilities in our products, even extremely minor ones, as defects for over two decades now. And we have always given them very high priority. To the best of our knowledge we've never had a remote exploit vulnerability, but even so we've gone so far as to scrap thousands of freshly pressed CDs a day before releasing them because I spotted a way to get root access through a tricky bit of business with shared libraries. (And that was for something spotted internally - no customer ever reported it.) The real question isn't whether to treat security vulnerabilities as a defect - of course you do - but - somewhat paradoxically - whether or not to treat them as security vulnerabilities. We were acquired some time ago and have now adopted (and adapted to) various more complex procedures typical of a large company. There's this little box you're supposed to check in our current bug reporting system that says "this is a security vulnerability". The problem is that checking that box fires up a whole lot of extra process that rarely helps and can actually hinder prompt resolution of the problem and getting the fix into customer's hands.
From your standpoint, your company treats vulnerabilities as defects. I think the author (McFeters) is making the point that most companies do not. Since he does work as a consultant for Ernst & Young, and has many times mentioned his background with web application pen testing, I think it's safe to say he probably has a pretty good grasp on how many do treat vulns as defects and how many don't.
I'm curious though, if your company treats vulns as defects, then do you punish or praise employees as the author suggests you should? Has your company implemented the much talked about Secure SDLC to catch vulns in the design phase?
SecureThroughObscure
Hello, this is Nate McFeters, the original author of the article.
Awitod quoted me as saying:
"The problem of course is I'm saying how the companies should handle them, and I have no authority at any of these places, save people actually valuing my ideas. Personally, I've done some development in the past, and there was the concept of defects. Your bonus would depend on how many defects were in your application at delivery time. These were feature-based defects, but shouldn't vulnerabilities be considered defects as well?"
And has responded with the following:
"So, the author freely admits he is neither a developer or a manager. If he was a developer he'd know that these are defects and everyone treats them as such."
To that, I respond by saying that I am no longer a developer or manager, although I did develop applications in both Java and C++ for around 6 years full-time, and managed a significant development effort of over 200K lines of code. Currently, I work at Ernst & Young's Advanced Security Center as a consultant, where I work with a lot of fortune 500 companies, mostly on addressing their web application security needs. I've performed deep source code assessments on applications over 2 million lines of code and been tightly integrated into a few companies development lifecycles. SO, that said, I truly have a ton of experience with this type of thing, and I'm actually surprised that so many people on the Slashdot list here have responded saying that they ARE treating these as defects, as this is NOT what I've seen from a majority of my clients and does not reflect what I've heard from others in my line of work with other consulting companies.
Awitod goes on to say:
"If he was a manager, he'd know that one of the surest ways to wreck a good shop is to start doing comp based on defects. Here is what invariably (in my experience) happens when a shop includes defect counts in there comp plans.
1. Relationships between Dev, QA, Product Management and Operations get worse because the terms 'defect' and 'bug' become toxic. In reality these things always exist in software. The last thing you want to do is create barriers to dealing with them. Making the acknowledgment of a defect cost someone money means you will have arguments over every one of them unless they cause an out right crash."
I don't agree that this is necessarily the case. I don't think you have to wreck relationships to encourage a positive competitive environment. This is exactly why I suggested moving to a positive culture that includes training for developers, a secure software development lifecycle that encourages challenging development and business to look at security issues throughout the entire development of an application, and most importantly, I actually suggested that rather than take the negative approach... reinforce positively.
For example, the top BU could get the biggest bonus, or a celebratory night out, or whatever... maybe it's just a trophy or championship belt they get to keep with their group, which could be a fun pride point. Positive competition can be astoundingly effective at managing a team.
Awitod continues:
"2. Culture becomes overly risk-averse - No one wants to take on difficult problems or blaze new territory. The smartest people will naturally pick the easiest work to minimize the risk of defects."
I disagree and feel this could be fixed quite simply. If you address possible security concerns in the design phase of the application build out, you have a pretty good idea of the threat threshold of the application. Applications that are tackling tough challenges or blazing new territory could be given higher scoring factors.
Think of this similar to how an Olympic Diver can perform a perfect jump, but still get a lower score than someone who just missed a tougher jump. In this case, strong teams will be striving to take on the tougher challenges. You could also put other factors into that as wel
Good to see a response from the original author on this. Awitod, don't confuse a difference of opinion with lack of experience.
I think that McFeters would agree with you, probably except for the part where you call him a "clueless writer dork". This is more often than not a symptom of the culture of a company, not of the developers. If you read McFeters' whole article, he comments on creating a more positive culture that rewards developers who catch bugs, etc.
This is the perfect example of why a lot of companies need a reality check. You can't hold devs under the fire for flaws that were introduced by the original design itself, which more often than not is driven by flawed business requirements.
SecureThroughObscure
You're looking at discovery of flaws at the wrong point... as most do. If you constantly find flaws by hiring pentest firms, you are in the wrong stage. You need to get Secure SDLC built into your development and actually try to catch these flaws in the design phase. This will considerably lower the cost and number of vulnerabilities.
If you constantly find flaws by hiring pentest firms, you are in the wrong stage. You need to get Secure SDLC built into your development and actually try to catch these flaws in the design phase.
That is great in theory, and might be true in the future, but you are missing the reality of the software development industry as it stands today.
1) Universities are not teaching software engineers about application security.
2) Most development organizations do not have leadership that understand the complexities and processes needed for secure software engineering.
3) Network Security training organizations like SANS teach courses around application security that barely teaches developers the skills needed to write secure applications. They still approach appSec from an operations point of view - where just like WAF deployment, you are too late or doing to little.
To really crack the AppSec nut, I recommend you approach this problem from several angles.
1) Start by (continuously) training your developers regarding application security. There are few firms that really do this well in a developer-centric way. Aspect Security ( http://www.aspectsecurity.com/ ) and Whitehat Security ( http://www.whitehatsec.com/ ) are 2 of the leaders in this field.
2) Begin building in-house application security pentest teams - this is a very different skill set than netSec pen testers. (You are right, you cannot just keep hiring pentest firms for the long haul)
3) Next do a risk assessment and catalog your applications current risk posture. Management appSec training is needed during this phase, as well.
4) Bring in a appSec pentest firm to assess your highest risk applications. Keep track of these results carefully so that fixes from developers can be tracked over time to verify that you are really reducing the applications risk posture over time.
Achieving application security excellence is a difficult process. And 3rd party application security training and application security pentesting (assessment) is critical on the path to success.
Horns are really just a broken halo.
Software companies don't even treat defects as defects.