Slashdot Mirror
Should Vendors Close All Security Holes?
johnmeister writes to tell us that InfoWorld's Roger Grimes is finding it hard to completely discount a reader's argument to only patch minimum or low security bugs when they are publicly discovered. "The reader wrote to say that his company often sits on security bugs until they are publicly announced or until at least one customer complaint is made. Before you start disagreeing with this policy, hear out the rest of his argument. 'Our company spends significantly to root out security issues,' says the reader. 'We train all our programmers in secure coding, and we follow the basic tenets of secure programming design and management. When bugs are reported, we fix them. Any significant security bug that is likely to be high risk or widely used is also immediately fixed. But if we internally find a low- or medium-risk security bug, we often sit on the bug until it is reported publicly. We still research the bug and come up with tentative solutions, but we don't patch the problem.'"
I did not RTFA, but I did read the summary. I did not hear his argument, I heard his conclusion repeated with more words.
Yes.
Short answer no with a maybe.
Is that you, Mr. Gates?
--- Asking inconvenient questions for over 30 years...
Exploit Chaining means that low risk holes can become high risk hole when combined. Patch them all. Patch them quickly.
If you mod me down, I shall become more powerful than you could possibly imagine.
We still research the bug and come up with tentative solutions, but we don't patch the problem. I can understand the point if it's to save time and money for other things, but if they are going to find a solution to the problem and time/money is already spent, then that is completely wasted if it isn't utilized. Plus you're risking the data by not closing a known hole or bug. Doesnt make sense.
You don't have to floss all your teeth, just the ones you want to keep.
As is too common, the ./ summary doesn't have the relevant portions of the article under discussion, so let me try to summarize the main points of their argument.
1. It is better to focus resources on high risk security bugs.
2. We get rated better (internally and externally) for fixing publically known problems.
3. Hackers usually find additional bugs by examining patches to existing bugs, so a patch could expose more bugs than fixes are available for.
4. If we disclose a bug and fix it, it just escalates the "arms race" with the hackers. Better to keep it hidden.
5. Not all customers immediately patch. So by announcing a patch to previously unknown to the public bug, we actually exponentially increase the chances of that bug being exploited by hackers.
It could work as well as the normal method, but if it catches on, it will mostly be used as an excuse to not do anything until publicly shamed. Call me cynical.
Hax-fu?
Seems like a pretty dumb move to me. You have the choice of, A: Patching immediately, costing you a few hours of time from a couple of your employees or B: Hoping that it won't be a big risk effectively betting a few hours of time against the possibility of a huge security breach and the corresponding bad press that comes with that.
Seems like a small patch wouldn't be that much trouble and would avoid much larger problems...
There are two kinds of fool One says 'This is old therefore good' Another says 'This is new therefore better'- Dean Ing
Yup, it's not about quality software, it's about money. Hardly anyone makes software anymore because they want to or because they like making quality software...they'll just do the bare minimum they can to maximize profit.
Basically ...
.... what the fuck?
#1. If we spend time fixing those bugs, we won't have as much time to fix the important bugs.
Translation: we put in so many bugs that we don't have time to fix them all.
#2. We give priority to any bugs that someone's leaked to the press.
Translation: we only fix it if you force us to.
#3. "Third, the additional bugs that external hackers find are commonly found by examining the patches we apply to our software."
I had to post that verbatim. They're releasing new bugs in their patches.
#4. "Fourth, every disclosed bug increases the pace of battle against the hackers."
Yeah, that one too. The more bugs they fix, the faster the
#5. If we don't announce it, they won't know about it.
Great. So your customers are at risk, but don't know it.
Pre-emptive disclosure works against the typical closed source company.
Option 1:
Exploit is published, patch is delivered really quickly. Sysadmin thinks, "Those guys at company X are on top of it..." PHB will say the same.
Option 2:
Unilaterally announce fixes, make patches available. Sysadmin doesn't bother to read the whole announcement and whines because it makes work she doesn't understand or think is urgent. PHB thinks "Gee company X's software isn't very good, look at all the patches..."
The market for secure software is small even smaller if you add standards compliance. Microsoft is a shining example of how large the market is for insecure, non-standard software.
http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
Examples:
Not likely to be fixed completely - In some ways, Windows is a security hole
Could be fixed if escalates - password strength and use
Should be fixed - Lack of any authorization requirements etc.
If you remember the Pinto car-b-ques, there is a money factor to think about. Since most standard computing systems are not life-critical, some bugs can be left for later. Some bugs you might know about but they are not in your code such as those shipped with the networking stack of the RTOS that you use for an embedded product. An insecure FTP client on an embedded machine that has no access to other machines or sensitive material is not terribly bad.
On the other hand, if the machine can be compromised and allow the attacker access to other machines... that needs to be fixed.
Support NYCountryLawyer RIAA vs People
Was not it GM, that lost millions of dollars a few years ago in a lawsuit brought by people (and their kin), whose car was rear-ended on a toll plaza and exploded in flames?
GM's arguments, that making the car's fuel-tank more protected was too expensive for the modicum of additional safety that would've provided, were — for better or worth — ignored by the jury...
In other words, you may not deem a security hole to be large compared with the expense of pushing out another patch, but if somebody gets hurt, and their lawyer subpoenas your internal e-mails on the subject, you may well be out of business.
In Soviet Washington the swamp drains you.
Why would a corporation choose to not release a patch for a known security vulnerability, even if it is minor? Wouldn't it be better PR to always release the patch before the exploit comes out? This sounds totally unethical to me. They are trying to take an ostrich approach to security: the bug doesn't exist unless the customer can see it.
Besides, aren't there liability issues with knowingly shipping a product with undisclosed defects? What if they underestimate the severity of a vulnerability? How can they be so confident that they have judged the severity correctly, when they are the ones who created the bug in the first place? This sounds a lot like brinkmanship to me, and I wouldn't want to be associated with that kind of company.
The one you hope for: Someone finds and public announces a problem. You team looks "quick to act" deploys a solution.
That other one: Someone exploits the bug to a degree you and your team never considered and your user community is devastated.
all known security bugs should be fixed, but low-risk non-public ones can be low priority. we cant expect any vendor to send a patch each and every time they find a security bug, but once they find one the next version they release damn well better have it patched.
The arguments only make sense if one's development methodology is purely reactive. The guy makes it sound like his entire software team is scampering around frantically putting out fires, and doesn't have any time to spare on any bug that hasn't reached three-alarm status yet.
I'd rather squash my bugs when they're unknown or barely known than race to hack together a fix when the whole world is screaming. But apparently that's just me.
I think developers and companies should think long and hard about how such policies would be received if the end-user were presented with them in plainspeak.
"Welcome, JoeUser. This is WidgetMax v2.0.3. As a reminder, this product may contain security holes or exploits that we already know about and don't want to spend the money to fix because we internally classify them as low-to-medium risk."
I'm not saying it's necessarily wrong -- budgets are finite -- but keeping policies internal because of how they would be viewed publicly is deceiving your customer, full stop. Also, these guys are setting themselves up as final arbiter of what's risky in code exploits. It makes one uncomfortable to think about.
Think of this in some other industry.
If Say a major car company found a systemic flaw in there product, they would be compelled to fix it and rework any product that had already shipped.
This is called a recall.
Something about Fight Club.... and some a * b Cost of a recall math.
A security hole is a bug, plain and simple. There's no excuse for deliberately not fixing a bug. Now, you can make an argument that if the bug's minor and not causing customer problems you should hold the fix for the next regularly-scheduled release, but that's about it. The argument that unannounced holes don't seem to be being exploited is particularly disingenuous. People aren't looking for exploits of holes they don't know about. It's not surprising, then, that few people are reporting problems they aren't looking for. What's more likely is that the small subset of crackers who can find unannounced holes are quietly using them for their own gain, keeping a low profile specifically to avoid having customers raising a stink and forcing the vendor to close the hole, and only after it finally goes public do they release their secret to the wider script-kiddie population since it's no longer of use to them.
If an automaker or toy manufacturer didn't issue a recall on a minor safety issue immediately, they'd get tons of bad press. But a software company can sit on just about any security bug indefinitely (I'm looking at you, Microsoft) and few people care.
I suspect 2 factors are at work here:
#2 probably won't ever happen industry wide, and until the public understands how much impact software security can have, they won't care.
We do the same thing. Every company has limited resources, and every decision is a usiness priority decision. So the decision is always between new features and old bugs.
Outside of terribly serious security holes, security holes are only security holes when they become a problem. Until then, they are merely potential problems. Solving potential problems is rarely a good idea.
We're not talking about tiny functions that don't consider zero values. We're talking about complex systems where every piece of programming has the potential to add problems not only to the system logic, but also to add more security holes.
That's right, I said it -- security patches can cause security holes.
It is our standard practice not to touch things that are working. Not every application is a military application.
I'll say it again. Not every application is a military application.
Your front door has a key-lock. That's hardly secure -- they are easily picked. But it's secure enough for most neighbourhoods.
So the question with your software is: when does this security hole matter, and how does it matter. If it's only going to matter when a malicious person attacks, then the question comes down to how many attackers you have. And if those attackers are professional, you might as well make their life easier, because they'll get in eventually in one way or another -- I'd rather know how they got in and be able to recover.
How it matters. If it reveals sensitive information, it matters. If it causes your application to occasionally crash, when an operator is near-by, then it doesn't matter.
There are omre important things to work on -- and many of these minor security holes actually disappear with feature upgrades -- as code is replaced.
I could see not waiting till your next regular patch, so as to avoid bringing it to the attention of the hackers.
But the rest of his arguments are pretty crappy.
excitingthingstodo.blogspot.com
Then running that software is not "practical". Any vulnerability is a vulnerability.
And "sufficiently complex" is defined as having "bugs and security holes". No. That just means that there is no such thing as "security". And we've been over that before.
Right. And enough of those "not particularly high risk" vulnerabilities can be linked together to crack your machine as surely as 1 remote root exploit could be.
"Best" in this case is being defined as "best for the company selling the software" and NOT "best for the people using that software".
What would be best for the users is the knowledge that there are fundamental security issues and that they might want to use a competitor's product.
Again, "not worth it" from the point of view of the vendor. Let's be clear on that.
How would you KNOW if there were already exploits in the wild? Unless someone was advertising them.
That approach means that an exploit can sit for years at the "unimportant" level
Yeah, that's the kind of support I want from my vendors.
From TFA:
"It's possible that if anti-virus software had never been created, we wouldn't be dealing with the level of worm and bot sophistication that we face today."
And if we didn't use antibiotics we wouldn't be seeing the current evolutionary pace of biological malware.
TFA presents some points for discussion, but this doesn't strike me as one of them.
Did I really just type 'biological malware?'
Regards.
...doesn't mean it is the key to success.
I really want to know what company the "reader" works for, so I can add them to my shit list. I don't want to support such abhorrent security practices.
And remember: Friends don't let friends buy Microsoft.
IANAL, but if a company suffers a significant financial loss due to a bug that the vendor knew about but did not patch, does that not open them up for big time law suits?
"Always forgive your enemies; nothing annoys them so much." - Oscar Wilde
Is President-Vice Richard B. Cheney's Spider-Hole.
I hope this helps the war criminal trial in The Hague.
Yours PatRIOTically,
Kilgore Trout
selling a product which you know does not meet specifications in order to derive a benefit is a crime called "fraud".
Trying to artificially inflate your bugfix ratings or trying to save money is a benefit.
I dont see how this company expects to evade legal CRIMINAL responsibility when someone is harmed because of a pre-existant security problem they knew about but did not disclose at the time of sale.
No one has a right to their *own* opinion. They have a right to the TRUTH.
1. Our programmers are trained to program perfectly
2. We spend our time fixing critical bugs, fixing medium and low security bugs slows down fixing the high priority bugs
3. Next priority (after critical bugs) is fixing publicly disclosed bugs
4. Hackers find bugs by examining patches so when we patch bugs that are not publicly disclosed hackers find out about them and then they become publicly disclosed bugs and our bug count goes up.
5. Our fixing vulnerabilities makes hackers smarter/better.
6. Most hackers only exploit publicly disclosed vulnerabilities. By not fixing bugs our customers are protected against most hackers.
No one is arguing that.
The discussion is about whether the attempt should be made to address ALL of those
Where did I say that?
They SHOULD be prioritized. No sense in trying to patch a local user, non-exploitable crash bug when you have a remote root vulnerability (with exploit).
But the system is only as secure as all of its components. While you may not believe that your vulnerability is important enough to patch, if enough people take that same approach, the vulnerabilities can be linked and your box will be cracked just as if you had left a remote root vulnerability unpatched.
That approach means that YOU are relying upon EVERYONE ELSE to cover your code by ensuring that THEIR code does not have any exploits. While you don't bother to patch your's.
..But on the flip side, not every bug can be exploited, even though it may seem severe. Every conceivable buffer overflow is not useable in practical terms.
I guess the point is, you have to prioritize.
And seriously, if you want software with guaranteed security or reliability, you probably don't want consumer grade stuff to begin with. (Just call your local nuclear power plant and ask them who wrote their process management software. )
I'll go with your reading. Thanks!
All problems are theoretical until they happen to you. Before they happen to you they fall into two categories:
It can never happen to you Probability = 0.0
It might happen to you Probability 0.0 x 100.0
The problem is that we don't know this when we hear of a problem. All we hear about is the theoretical problem and the probability of a theoretical problem being theoretically true = 100.0%. If we had a way to neatly classify vulnerabilities into both the probability of occurence and the probability that something of a given 'oh-shit'-edness happening then we could easily ignore problems that are more expensive to fix than the risks they present. Since we can't really do risk analysis we're stuck with trying to do everything, I guess, or not. I'm not sure. Maybe it doesn't matter much.
When you work people 80+ hour week you get a lot more bugs in the code.
When you rush things out to meet a clueless PHB dead line things get passed over.
When you cut funding some times you don't have the hardware to do full testing and you end up with test code on the sever or a desktop turned in to a test sever.
When you waste the codes time on crap like TPS reports useless meetings to tall about way things are running late that does not help.
The problem with vendors closing security holes is, they often can't keep up with the volume of it.
In the case of MS, how often do they close one hole only to open up another. I don't want to throw OS' around, but look at team OpenBSD, regardless of the smug attitudes, you have to give Theo and his group credit. They don't release for the sake of keeping up with the Jones'. They're methodical and accurately screening and scrutinizing what their OS does, what its supposed to do and how it does it.
The issue with vendors are, they're in a race to meet stockholder/shareholder demands and are often releasing whatever they can to meet sales forecasts. This leads to shoddy designs and implementations. If you ask me, some of these vendors should be getting fined by governments based on the severity of their holes. I'm sure with that kind of pressure, they'd be more reluctant to release cruddy programs as opposed to hurrying something insecure out the door.
If I were a vendor and all of the sudden I was getting fined for not taking security seriously, I would put on a big old about face and audit the hell out of my code. But what incentives, qualms, repercussions face vendors now? None. People will bitch about MS' security yet go out and buy Vista. How about people start taking things very serious and start suing some of these vendors... "My personal information was compromised because of shoddy program X class action lawsuits." Hit em where it hurts, eventually they will either listen or go broke.
Infiltrated dot Net
The place I worked for was a security company. They had no automated unit testing and never analyzed for intrusions. You'd be shocked to find out how many holes exist on devices people depend on to keep them safe. The employees took it upon themselves (subverted authority) to patch our product. Security problems, even on security hardware, were not "priority" issues.
We too "trained" our coders in the art of secure programming. The problem, of course, is that we were also training them in basic things like what a C pointer is and how to not leak memory. Advanced security topics were over their head. This is the case in 9 out of 10 places I've worked. The weak links, once identified, can't be fired. No... these places move them to critical projects to "get their feet wet."
At the security giant, training only covered the absolute basics: shell escaping and preventing buffer overflows with range checking. The real problem is that only half of our bugs were caused by those problems. The overwhelming majority were caused by poor design often enabled by poor encapsulation (or complete lack of it).
There were so many use cases for our product that hadn't been modeled. Strange combinations of end-user interaction had the potential to execute code our appliance. Surprisingly, our QA team's manual regression testing (click around our U.I.) never caught these issues, but did catch many typos.
I don't believe security problems are inevitable. I've been writing code for years and mine never has these problems (arrogant, but mostly true). I can say, with certainty, than any given minor-version release has had 1,000's of high-quality tests performed and verified. I use the computer, not people... so there's hardly any "cost" to do so repeatedly.
I run my code through the paces. I'm cautious whenever I access RAM directly. My permissions engines are always centralized and the most thoroughly tested. I use malformed data to ensure that my code can always gracefully handle garbage. I model my use cases. I profile my memory usage. I write automated test suites to get as close to 100% code coverage as possible. I don't stop there. I test each decision branch for every bit of functionality that modifies state.
Aside from my debug-time assertions, I use exception handling quite liberally. It helps keep my code from doing exceptional things. Buffer overflows are never a problem, because I assume that all incoming data from ANY source should be treated as if it were pure Ebola virus. I assume nothing, even if the protocol / file header is of a certain type.
Security problems exist because bad coders exist. If you code and you know your art form well, you don't build code that works in unintended ways. Proper planning, good design, code reviews, and disciplined testing is all you need.
That approach allows hackers to exploit known-about but unreported (and therefore unfixed) loopholes potentially for ages.
Please give me the name of this guy's company so I can avoid all their products.
They say: if you know about it, you're obliged to fix them. And then you kick your QA department's butts around the corridor several times. If your customers are your software testers, then you're business model is likely corrupt. And while there are a number of coders that will complain that it was the libs, or the other guy's fault, ultimately, a responsible organization takes ownership of their faults, just like humans should.
---- Teach Peace. It's Cheaper Than War.
Not patching security flaws no matter their "level of criticality" is like standing in the middle of some railroad tracks that's lightly used. Sure you might stand there for a long time before a train comes along but when it does your toast.
My karma is not a Chameleon.
As a developer, while I'd like to fix all the bugs in the system, the truth is there will always be a few that require architecture changes or impact large sections of code, and as such tend to persist in the application. Since these bugs tend to require a large time investment for almost no perceptible gain for the end-user, they are often ignored in favor of devoting time to new features.
Should all bugs be fixed? Yes. Should all bugs take precedence over new features? No. This is not to say developers do not try to fix long-standing issues, just that the churn rate for bugs tends to be rather constant. (Once an older bug is fixed, a new one takes its place.)
affredd... be veddy effredd.
In code space, noone can hear you scdream...
This is a GOOD case of "chain" smoking...
Maybe they need to debug their approach? Is anyone else bugged by this?
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
... fixing their bugs instead of studying whether to fix them they would have a better product. It reminds me of a joke http://www.infojokes.com/index.php/archives/10184
Are you willing to indemnify your users for any and all losses suffered by them due to a flaw/bug which you knew about but chose not to patch? If not, then patch it!
'The tyrant will always find pretext for his tyranny.' - Aesop's Fables
4-inch crack in a chastity belt by applying multiple layers of Armor All, or maybe Poly-glycote...., butt, on the other hands....
(CAPTCHA: DISCREET)
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
The problem is that if you've discovered a security hole; chances are someone else has as well. Just because a problem hasn't been reported to your company doesn't mean that it is unknown.
History shows that there are lots of black hats that will sit on security breaches/expploits/bugs/etc and exploit them for their own end rather then reporting them to a company. Breaches in security should be patched as soon as they are discovered. If 1 person found the bug/hole/exploit/whatever, that means another person can find it as well. There's nothing there about having to report it to a vendor once found by either person.
One of these days i'm going to find this 'peer' guy and reset HIS connection!
This is not entirely true. It is not always feasible to change your software just like that. Processing patches can be a lot of work for the customer, as their IT department might wish to test the patched version on test systems. Furthermore a patch may impact other parts of an application, especially if a bug resides deep in the framework and the application is complex.
In such a case it really is not bad to consider fixing a low-risk bug in the next "normal" release when a usual full QA test has verified that no other stuff has, or seems to be broken.
Leaving a high-risk, or highly irritating bug be will cost you customers. But daily dumping patches on your customers ICT department will also not endear you to them; and changing functionality or accidentally introducing new bugs will not endear you to the end users.
It's always a difficult decision to make, and you never make the right one:)
By the way, I like making software, and I like to make quality software. It's all about the money, but consistently delivering bad or buggy software will cost you your customers. It does not (always) pay to be cheap, and I believe most companies know that.
Just as often the reverse applies. A bug often shadows other bugs. Take away the main bug and there's just another right behind it which might even be worse. This is why you don't just "shoot from the hip" when fixing bugs.
Engineering is the art of compromise.
If it aint FOKE don't BRIX it? EESHSH
I suppose that is why some software companies use the S/W equiv of Poly-Razz-Ma-Tazz... overwhelm the user with a plethora of "features" so that metrics look better when ONE otherwise major bug would stand out in a small-feature program/app. But, then many customers (and devs/publishers) cannot see the forest for the trees.
Previously: "Linux... Toward the Sunrise..." Now: "Linux... Toward the-- No, now, part of Every Sunrise"
>>We still research the bug and come up with ****tentative**** solutions, but we don't patch the problem.
They come up with a tentative solution, but they don't spend the time to do full testing on it unless it's a critical security hole. Why? As mentioned, they prefer to:
1) Use limited resources to focus on finding critical problems
2) Not introduce new code to a known system unless necessary
3) Not put their real-world, paying-customers-who-don't/can't-patch in danger.
Again, the letter makes clear that they do patch the most critical things, and anything that is public (and thus likely to be exploited).
If the hole becomes known, they can issue a fix more quickly because they already have a starting point.
Yes, yes they should patch them all. Personally it'd eat away at me knowing I could spending a few minutes, hours, or days to fix a vulnerability in my software. I don't think I could take pride in what I do if I just leave crap like this around because I don't have to fix it and don't think it's important unless someone finds it publicly. I'm glad they fix the HIGHs (however they rate this.. who knows?) and the publicly disclosed ones. But why not fix the small ones as you find them? It's a little bit of embarassment every time an issue is found. This is one less piece of embarassment. However, maybe it's the quasi-perfectionist in me, I couldn't imagine not fixing this stuff.
What exactly is a security hole and what exactly is a feature?
Simple truth is - for any software with sufficient amount of users there will be users that will manage to find even most strange "features" and often exploit to their advantage (as in - it benefits them actially).
As long as you are facing the unknown (users that might actuall think that any particular issue is a feature and use them) versus known ("yes, there seems to be a problem with IP parsing if using nonstandart syntax, but nobody has complained") those two has to be weighted quite carefully.
They should absolutely patch bugs when discovered, regardless of classified severity. They should take the OpenBSD approach and regularly and aggressively audit their code. I think customers that have paid good money for a product, deserve one as bug free as possible. OpenBSD is an OS that you get for free that has far fewer flaws and is proactive about letting its users know when a bug does crop up. If a free/open source project with an even tighter budget, there is absolutely no reason a commerical vendor cannot do the same.
The article and responses miss an important point: patches of any kind are risky! And not just because they might introduce a new security flaw, but more generally because they may break some feature or another. In applications with millions of lines of code, and where the cost of doing a patch release amortized over all customers is millions of dollars, it can make lots of sense to just roll a fix into the next planned upgrade release. That way you get a complete Q/A and customer beta-test cycle to increase the confidence level of the fix.
I RTFA (I must be new here;) and I'd like to know what shitty half-assed company the "reader" works for so I can avoid his company's software like the plague? God damn it, if you sell me software and you find a security hole there's a damned good chance that the black hats have found it too, and there's no way in hell they're going to disclose it to anybody. They're going to exploit it until your company finds it and patches it.
I know this won't sit well with most slashdot readers, but we (in all our countries, or at least one or two large ones) need some liability laws. If I buy your software and it needs patching, patching your poorly designed buggy insecure piece of shit should be on YOUR dime. If a defect that can cause a fire or accident is found, GM pays for it. Software companies should do the same for security bugs, and I would include bugs that can result in loss of data as well.
If your shitty software causes me a million dollars in lost data, I should be able to win a lawsuit against you for a million, plus legal fees, and if it turns out you knew about the flaw and sat on it I should be able to get punative damages as well.
Now, I know some of you halfassed so-called "software engineers" are going to whine about how complex modern software is, and my answer is "tough shit". Cars have literally hundreds of thousands of interlocking and moving parts, yet my last car was twenty years old when I got rid of it. Your software is NOT more complex than a modern car!
The difference between cars and software is the car's engineers are actually real live engineers who know WTF they're doing. Software bugs should be rare, and the only reason they're not is because we users allow it.
-mcgrew
You know something, I was planning on rebutting, but you're entirely correct. But you make it sound like a bad thing. Some of our clients are educated and experienced in the technical world. Many of them choose to see the forest from above -- and so they demand a taller tree first.
When given the option between a week of effort to fix a security hole (for free), or a week of effort to build a new feature (for cost), not all but most of our clients prefer the latter. They would rather grow their business (even on an unstable balancing act) than safe-guard their business. And all the advice in the world doesn't seem to be able to change that.
I guess it's the typical/traditional backup advice. I have yet to meet a client that is willing to spend the time and effort to develop a reliable backup routine (not even a restoration routine). They'd rather run the risk of losing all of their data, and dealing with the headaches and fall-out, than to spend money to prevent something that may never occur.
I can respect that -- hell I make the same decisions every day. And sometimes I even make those decisions with other people's data. But that's the very same risk/reward decision that every business entrepeneur makes. So in the grand scheme of things, I guess it's insignificant compared to most of the other decisions in running a business.
Just off the top of my head, I'd say that firing an employee is a bigger risk than worrying about hard drive failures.
>Solving potential problems is rarely a good idea.
It worked for OpenBSD, though conceivably they could have gotten the same results with less labor.
Their policy was to audit code looking for problems, and then fixing every problem they found without even checking whether it was exploitable.
Interestingly, one result was that OpenBSD became unusually difficult to crash.
Not many projects are willing to set up their priorities the way the OpenBSD team has, and there are reasons.
And that explains the massive growth of Linux and other Open Source apps.
It's "bullshit" to you, but it's very real to the customer. And the customers are moving because of it.
hi im from delhi, u should listen when i tell you how it is.
Odds are, since you type like a 14 year old texting someone - you probably don't know how it is. Call me after you make your first few college loan payments. kthxbye.
Don't assume we write crappy programs.
Don't have to assume. I'm actually in the industry and I've dealt with outsourced code. It IS crappy.
Don't assume you have greater brains. We are the country that invented Mathematics
There are a few greek fellows I read about who might want a word with you on that.
And the accent..ugh...we have the most neutral of all accents.
To YOU, maybe. Because everyone around you has the *same freaking accent*. Of course you find it agreeable. To the rest of the world it sounds like you're talking through a mouthful of pudding.
Don't u want a better quality of life too? Isn't it why you go and bombard every country you find oil in?
How has bombing anyone made our lives better? Did it make gas prices go back down? Have they lowered my taxes? No, both times. Gas is at an all time high, and taxes to support all this are astronomical. I much prefer a peacetime economy, thank you very much.
Be thankful of what you have and wait for the time when you would be fighting with each other to come and work in India.
Not bloody likely. I'd quit the field if I had to move to India.
Weaselmancer
rediculous.
I'm shocked by how many people answer this with an unqualified "Yes." That's not realistic at all.
Here's an example. An app asks for your password. That password gets written to memory for a period of time. During that time, the page containing the password may get swapped to disk. The system may then crash or lose power, leaving the password on disk. I could then boot into another OS, read the swap file, and recover your password. Unlikely, but possible.
There, I "found" a security hole. Want to patch it? You could try to make every app that uses a password store it in wired (not swappable) memory - but performance will suffer (and good luck doing that in every app). You could also dynamically encrypt all data that gets written to the swap file, and decrypt it when read, at the cost of performance again.
Are you willing to trade performance in every app to defend against this mostly theoretical vulnerability? Probably not. Security is about tradeoffs. Welcome to the realities of software development.
Has this guy figured in the time to re-investigate bugs when they get brought up again? Chances are if they blew it off in the first place they didn't take the time to document what was at fault. This would create a huge re-investment time for every bug that is reported by a vendor/hacker later on down the pipeline. Maybe he should compare the time re-investigating against the time to just patch the bug. I bet they are similiar, but the latter would give them a better, more secure product.
-Skeeterbug
So, Mr. Corporate Executive - your customers are at risk, you know it, and you're not going to do anything about it?
Oh, that's right - not until there's a public release of information about the security holes from someone else. Go ahead and wait for the media report - but don't be surprised if what they say isn't what you were expecting.
And you might want to check with your attorney about your liability for the losses your customers incur - because you knew about the risk and DID NOTHING to correct it or advise your customers.
Let me get this straight.
You are saying that, since most businesses and most people prefer risky practices that let them move faster, or with fancier stuff, to practices that secure what they have but won't move them forward as quickly, the risky practices must be better?
[sigh]
Yes, most people do it. Yes, I have been lax at back-ups myself, and often regret my failure to back up only after I lose important files (preserved on the internal hard drive of a dead computer--or corrupted on that hard disk--or saved on a disk in a proprietary format which my new computer won't read). But just because most businesses don't believe in back-ups doesn't mean back-ups are overrated.
There is a fine line between recklessness and courage... -- Paul McCartney
For commercial companies, it's all about PR and sales (since they've disclaimed any liability whatsoever anyway). A high number of bugs fixed can mean that either your product is a pile of crap to begin with, or that someone is doing a helluva job securing the code. To the public that's completely indistinguishable. The only test is the test of time - if you're still stomping out bugs, eventually your codebase will have much fewer bugs and less exploits. By the time you decide to play catch-up, hackers will first find your obvious bugs, then your almost obvious bugs, then your easy bugs... it'll take forever to catch up. For those looking at the next quarterly earnings, they don't care about that at all though. So by all means, play the FUD game instead of actually fixing bugs. When you fall you might find it's a loooooooong way down.
Live today, because you never know what tomorrow brings
It's also possible that "consumer" or more precicely, "off the shelf" (either comercial or free open source) might be, on average, more secure and sport fewer bugs than a given bit of custom software. There seems to be a wide variance in quality in all these categories, and overlap. The forces at work in favor of COTS or FOSS vs. Custom Software include a larger pool of users who are more likely to find problems, better funding, and a convergence on a more rational feature set or design that might come from more sources of input and an expected product lifecycle involving many customers over a long period of time. (This may not always lead to better design and higher quality, but probably does sometimes).
If you mod me down, I shall become more powerful than you could possibly imagine.
Risk is not fixed. Things change. One of the reasons to fix low (and medium) risk security bugs is so that they are mitigated before they can possibly become high risk security bugs.
I would bet that they have no process to re-evaluate bug risk on a regular basis. Only when a bug blows up in their face do they ask "How come this low risk security bug just bit us in the ass?". I doubt that think: "I wonder if there are other low risk security bugs that are no longer low risk?
His basic argument is that all the low level patching does is to accelerate the arms race between software makers and black-hats. Frankly, I think his argument is flawed. There are plenty of people out there who know how to exploit all kinds of things. Just because some of us choose not to do so doesn't mean that we don't know how. If *I* can find things and figure it out, I'm sure that other people can as well.
If you don't know that they're being exploited because you're customers don't know to look for it. That's like saying your security is good because you've never been hacked.
2 cents,
Queen B.
HDGary secures my bank
Don't you mean without slash (QA)?
Ant(Dude) @ Quality Foraged Links (AQFL.net) & The Ant Farm (antfarm.ma.cx / antfarm.home.dhs.org).
Is somewhat nuanced. I don't buy the argument in the story, but I don't buy the "fix every possible security bug as soon as it is discovered" argument either.
When you are developing software, there are always going to be bugs, security shortcomings (i.e. areas where the architecture is weak on security) and the like. In all cases, one must prioritize which bugs to fix.
I would give priority to bugs based on a number of criteria including:
1) How severe is an exploit? What can be done with it?
2) What is the overall effect on security from this particular bug?
3) Are there other mitigating factors? Any aggrevating factors?
4) How likely is it that this will be exploited?
Now, a few things I do not condone:
1) Having a policy which says "you leak this to the press and we will fix it." This encourages zero-day exploits.
2) Having a policy which says "publically noted bugs will be given priority." This encourages zero-day exploits.
A saner policy is, when doing triage, to evaluate the bug, its security, what work-arounds are available, etc. Publically document what is required to properly secure the application, and then triage against the implementation best-practices. Sure, where possible, one should be abe to fix as many problems as possible, but at least this way you can truthfully say "if you followed our documentation, this won't affect you."
In short, I entirely disagree with the article's argument. They are misapplying their limited resources to encourage security problems.
LedgerSMB: Open source Accounting/ERP
Should Vendors Close All Security Holes?
Yes. Just like you should obey ALL TRAFFIC LAWS.
Next?
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Outside of terribly serious security holes, security holes are only security holes when they become a problem. Until then, they are merely potential problems. Solving potential problems is rarely a good idea.
NONSENSE. Maybe it isn't a "good idea" to your manager whose only care is a deadline, but that's the exact opposite of what any rational person would do. Solving things when they are merely potential problems is the ideal time to solve them! If you wait to fix a bug until it causes problems, then you've waited to fix a bug until after it has hurt your customers. I bet they won't agree that you did things in the right order, waiting until it was no longer "potential".
Do you work for the Louisiana Corp of Engineers? "Well the levies are a potential problem, but I think it's better for us to wait until they are an actual problem before we fix them. That should be the smart thing to do." Just compare the cost of repairing New Orleans to the cost of fixing the levies to see just how much of a "good idea" that was.
I could go on and on and on for days and days with countless examples from software and Real Life of just how stupid it is to wait for something you know is a potential problem to turn into a real one. But so long as its the customer who pays the price for this attitude, I guess it works, eh?
We're not talking about tiny functions that don't consider zero values. We're talking about complex systems where every piece of programming has the potential to add problems not only to the system logic, but also to add more security holes.
That's right, I said it -- security patches can cause security holes.
Oh noes, what a terrible secret! That's why you take the lower priority bugs, fix them, and then spend the time to test the fixes to see if they introduce new bugs and release them in a periodic patch. Yes I know you can't prove you haven't introduced bugs, blah blah, you can't for any other checkin either. If by your own estimate the bug isn't urgent, then you can take the time to check your fix at least. Frankly I'd be more worried about the patch you have to hastily throw together when one of your "potential" problems becomes a real one and your customers are getting pwnzored and are demanding a fix yesterday. That's the fix that is more likely to introduce bugs, and it's a direct consequence of your "don't worry about potential problems until they bite us in the ass" idiocy.
Your front door has a key-lock. That's hardly secure -- they are easily picked. But it's secure enough for most neighbourhoods.
If there was a device that could automatically attempt to lock-pick any lock with a known vulnerability, and it could check thousands of locks per second, you better believe I'd be out to buy the best damn lock I could find, and upgrade every time a vulnerability was discovered in my existing lock. And I'd be pissed as hell if I found out my old lock vendor had known about an issue but didn't do anything until after my house had been robbed. Door locks serve as a deterrent because even if they are relatively easy to pick that at least takes time and effort for every lock. Computers allow the lock to be picked once and then every lock of that type can be opened with essentially zero effort. Thus why there are tens of millions of Windows zombies out there.
So the question with your software is: when does this security hole matter, and how does it matter. If it's only going to matter when a malicious person attacks, then the question comes down to how many attackers you have.
Once again, no. Remember computers? Automation? The question comes down to is there a single attacker on earth who knows the vulnerability. If the answer is "yes", then that one hacker can exploit every single one of your customers.
And if those attackers are professional, you might as well make their life easier, because they'll get in eventually in one
The enemies of Democracy are
When given the option between a week of effort to fix a security hole (for free), or a week of effort to build a new feature (for cost), not all but most of our clients prefer the latter. They would rather grow their business (even on an unstable balancing act) than safe-guard their business. And all the advice in the world doesn't seem to be able to change that.
I guess it's the typical/traditional backup advice. I have yet to meet a client that is willing to spend the time and effort to develop a reliable backup routine (not even a restoration routine). They'd rather run the risk of losing all of their data, and dealing with the headaches and fall-out, than to spend money to prevent something that may never occur.
Oh, okay, I see. Your customers are idiots, and you are merely providing them with the quality of software they deserve. If you just want to exploit the foolish for minimal effort, that's fine for you, just don't act like your policies are actually how software should be developed.
The enemies of Democracy are
Option 3:
Blackhats have the exploit for the problem the vendor is sitting on and extort the shit out of any company they can with it.
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
I can rebut a few of those claims.
... allow the lock to be picked once and then every lock of that type can be opened with essentially zero effort. "
"If there was a device that could automatically attempt to lock-pick any lock
There is. It's called a lock-pick. And every single lock that uses a standard key -- i.e. not an electronic key -- can be picked in under ten seconds. It's as easy as carding a door that doesn't use a deadbolt. We're talking seconds, and we're talking easy, by anyone, any house. Your garage door too. And your keyless entry system for your car. Hey, my car remote opens my friends garage door -- what are the odds?
"The question comes down to is there a single attacker on earth who knows the vulnerability. If the answer is "yes", then that one hacker can exploit every single one of your customers."
That's not the question, but it's close. The question is: "is there a single attacker on earth who WILL TAKE ADVANTAGE OF the vulnerability".
I think we can all agree -- well, scratch that. I'm asking. From a philosophical perspective, if you could guarantee that no one will do a particular something, what would be the benefit (i.e. direct benefit, not recreational) of ensuring that no one does?
There are plenty of aspects of society that are not regulated for the sole reason that they just don't happen. No one suggests that we sit down and devise laws for everything that could happen. In fact, we wait until someone does something, and then we rule on it, decide if it should or should not be illegal, and work towards an eventual law -- sometimes. And that's after people have been robbed, defrauded, killed, injured, or otherwise upset.
Same goes for software. There is an endless supply of potential threats, and limited resources with which to prevent them. Some solutions require time, others money. And some solutions have performance hits which make the solution a bigger problem always. I'm thinking a system that processes tasks. Insecure, ten tasks per second, secure five tasks per second. Unfortunately, there are eight tasks that need performing. Two machines could handle the load, but then they require twice as much maintenance -- which blows profit margins.
I'm not saying that planes should fall out of the sky, and I'm not saying that every problem should go uncorrected. I am saying, however, that planes do fall out of the sky. That there have been many successful shuttle launches, and three tragic ones. How many people think that only those three had shortcuts taken? Spin doctors are good. The point is that risk goes with reward. Like everything else, when you incur risk in the name of reward, the reward goes up. It a competative world, that's a business decision.
But hey, those are two very different lifestyles. I run a business where I've attempted to take no risks, offer great customer service, be responsive 24/7/52 because I prefer to sleep well at night and I love my ethics. But for the amount of effort that I've put in, I could have started ten business, run each one with high risks, not provided any service and just hoped for clients that tolerated it. All in the hopes of failing nine times and pulling a large profit on the tenth.
Human exploration has always been a high-risk venture with people dying before finding land. Human medicine has (typically) been a very safe venture, with the patient being put ahead of research.
My ultimate point is that "potential problem" does not mean "eventual problem". 90% of potential problems never amount to anything. So outside of dangerous cases, highly influencial situations, and drastic loss scenarios, it settles down to a business decision.
I think most here would agree that Microsoft has chosen to build new features in place of repairing bugs -- certainly not exclusively, but new features are released while existing bugs remain. Certainly Microsoft makes more money than I do. So clearly, it's a somewhat successful strategy.
I don't think that my clients are idiots. They understand the situation and choose to take the risks. That's not idiotic, if anything that's adventurous. It is a gamble, every time, but to date, most of them have paid off. Some have resulted in bad weekends and extra costs, and even a lost customer or two. But my understanding is that they've gained more customers by implementing requested features for potential customers than the number lost due to bug-related problems.
But that's the business trade-off. Not everyone is a good (i.e. profitable) customer.
Every time I see your sig, I do a second take. That should be "umount".
Graham
Until a few years ago, a few people would have called a buffer overflow bug caused by a malformed data file a low security bug. Few have imagined that a widespread use would be common. Sure, everyone knew that you could run code that way, but it would require someone with very good knowledge of the inner working of a processor to pull that stunt off, and it would work ONCE, since the user would usually immediately find out, at the very least, that the file "doesn't work" and would neither spread it nor use it again.
With the internet and its use to distribute malware, this changed significantly. First of all, there are code packages now that you only have to slap onto the exploit. Second, with the net you can spread this problem billion times instantly.
Who tells you that what you perceive now a "low security bug" won't be a serious threat with the advent or the general adaption of some technology? Sure, one could say that you'll have the next gen product out the door before that happens, but those bugs have existed for generations of products. Looking back through some of the thusly exploited products, you will find that this bug existed already in Versions of the program for Win95 and even Win3.1.
There is no such thing as a "low security bug". A low security bug is just a bug that didn't get used creatively yet.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
The whole "responsible disclosure" and "it's not public" crowd is ignoring the oldest, most boring and just a tiny bit important fact of the security world: Bad guys don't publish their 0days. They keep them for themselves or their group. The "0day" you read on bugtraq today is old news for many of the people you don't want to exploit it.
So not patching a bug because it's not (yet) public means giving priority root access to bad guys. If that's your philosophy...
Assorted stuff I do sometimes: Lemuria.org
As Bruce Schneier would say, security holes ultimately aren't a security problem as much as they are an economical problem, and this is a perfect example: the vendor does not care about whether their customers are secure; they only care about how acknowleding holes, putting out patches etc. will affect their bottom line.
The solution to this is to make sure that unpatched security holes DO affect their bottom line; for example, there could be a fine for leaving known holes unpatched intentionally. Once you shift the financial burdens to those in a position to fix the problem, the problem WILL get fixed, but if you don't do that... good luck; you'll need it.
butter the donkey
Don't forget about business costs people. Changes to software cost money. The smallest code change must be extensively tested in QA, and a software release can have hundreds of potential fixes/features that are queued and must be prioritized.
If this is the kind of software you work on, then yes, this solution will work.
Patrick Doyle
I mod down every jackass who puts his moderation policy in his sig. Oh, wait a sec....
In case of big data structures (in some of the cases I've had to deal with: hundreds of thousands of objects, built gradually and modified heavily during several hours) finding the exact sequence that causes the inconsistency can be a nightmare.
Did you mean to type "hundreds OR thousands of objects"?
I'm kinda dubious that even M$FT Vista has hundreds OF thousands of objects [i.e. hundreds OF thousands of instantiations of classes] when it boots up. Or that Vista even ships with hundreds OF thousands of classes capable of being instantiated.
Even the idea of "thousands" of objects makes the hair on the back of my neck stand up on end...
Or maybe you use the word "objects" to mean something other than "instantiations of classes"?