Slashdot Mirror
Bug Bounties Don't Help If Bugs Never Run Out
In 2007 I wrote:
It's virtually certain that if a company like Microsoft offered $1,000 for a new IE exploit, someone would find at least one and report it to them. So the question facing Microsoft when they choose whether to make that offer, is: Would they rather have the $1,000, or the exploit? What responsible company could possibly choose "the $1,000"? Especially considering that if they don't offer the prize, and as a result that particular exploit doesn't get found by a white-hat researcher, someone else will probably find it and sell it on the black market instead?
Well, I still believe that part's true. You can visualize it even more starkly this way: A stranger approaches a company like Microsoft holding two envelopes, one containing $1,000 cash, and the other containing an IE security vulnerability which hasn't yet been discovered in the wild, and asks Microsoft to pick one envelope. It would sound short-sighted and irresponsible for Microsoft to pick the envelope containing the cash — but when Microsoft declines to offer a $1,000 cash prize for vulnerabilities, it's exactly like choosing the envelope with the $1,000. You might argue that it's "not exactly the same" because Microsoft's hypothetical $1,000 prize program would be on offer for bugs which haven't been found yet, but I'd argue that's a distinction without a difference. If Microsoft did offer a $1,000 prize program, it's virtually certain that someone would come forward with a qualifying exploit (and if nobody did, then the program would be moot anyway) — so both scenarios simply describe a choice between $1,000 and finding a new security vulnerability.
But I would argue that there are certain assumptions under which it would make sense not to offer a cash prize program — and, in keeping with my claim that this is equivalent to the envelope-choice problem, under those assumptions it actually would make sense for Microsoft to turn down the envelope containing the vulnerability, and take the cash instead. (When I say it would "make sense", I mean both from a profit-motive standpoint, and for the purposes of protecting the security of their users' computers.)
On Monday night I saw a presentation put on by Seattle's Pacific Science Center "Science Cafe" program, in which Professor Tadayoshi Kohno described how he and his team were able to defeat the security protocols of a car's embedded computer system by finding and exploiting a buffer overflow. That's scary enough, but it was more interesting how his description of the task made it sound like a foregone conclusion that they would find one — you simply sink this many person-hours into the task of looking for a buffer overflow, and eventually you'll find one that can enable a complete takeover of the car. (He confirmed to me afterwards that in his estimation, once the manufacturer had fixed that vulnerability, he figured his same team could have found another one with the same amount of effort.)
More generally, I think it's reasonable to assume that for a given product, there is a certain threshold amount of money/effort/person-hours such that if you throw that much effort at finding a new security vulnerability, you will always find a new one. Suppose you call this the "infinite bug threshold." Obviously the amount of vulnerabilities is not really infinite — you can only do finitely many things to a product in a finite amount of time, after all — but suppose it's so close to infinite as to make no difference, because the manufacturer would never be able to fix all the vulnerabilities that could be found for that amount of effort. I'm sure that $10 million worth of effort, paid to the right people, will always find you a new security vulnerability in the Apache web server; the same is probably true for some dollar number much lower than that, and you could call that the "infinite bug threshold". On the other hand, by definition of that threshold, that means that the amount of vulnerabilities that can be found for any amount of money below that, will be finite and manageable.
(I'm hand-waving over some details here, such as the disputes over whether two different bugs are really considered "distinct," or the fact that once you've found one vulnerability, the cost of finding other closely related vulnerabilities in the same area of the product, often goes way down. But I don't think these complications negate the argument.)
Meanwhile, you have the black-market value of a given type of vulnerability in a given product. This may be the value that you could actually sell it for on the black market, or it may be the maximum amount of effort that a cyber-criminal would invest in finding a new vulnerability. If a cyber-criminal will only start looking for a particular type of vulnerability if they estimate they can find one for less than $50,000 worth of effort, then $50,000 is how much that type of vulnerability is worth to them.
Now consider the case where
infinite bug threshold > black-market value
This is the good case. It means that if the manufacturer offered a prize equal to the black-market value of an exploit, any rational security researcher who found a vulnerability, could sell it to the manufacturer rather than offering it on the black market (assuming they would find the manufacturer more reliable and pleasant to deal with than the Russian cyber-mafia). And we're below the infinite bug threshold, so by definition the manufacturer only has to pay out a finite and manageable number of those prizes, before all such vulnerabilities have been found and fixed. I've made a couple of optimistic assumptions here, such as that the manufacturer would be willing to pay prizes in the first place, and that they could correctly estimate what the black-market value of a bug would be. But at least there's hope.
On other hand, if
infinite bug threshold < black market value
everything gets much worse. This means that no matter how many vulnerabilities you find and fix, by the definition of the infinite bug threshold there will always be another vulnerability that a black-hat will find it worthwhile to discover and exploit.
And that's the pessimistic scenario where it doesn't really matter whether Microsoft chooses the envelope with the vulnerability or the envelope with the $1,000, if the infinite-bug-threshold happens to be below $1,000. (Let's hope it's not that low in practice! But the same analysis would apply to any higher number.) If the black-market-value of a bug is at least $1,000, so that's what the attacker is willing to spend to find one, and if that's above the infinite-bug-threshold, then you might as well not bother fixing any particular bug at that level, because the attacker can always just find another one. It doesn't even matter whether you have a prize program or not; the product is in a permanent state of unfixable vulnerability.
At that point, the only ways to flip the direction of the inequality, to reach the state where "infinite bug threshold > black-market value", would be to decrease the black market value of the vulnerability, or increase the infinite bug threshold for your product. To decrease the black market value, you could implement more severe punishments for cyber-criminals, which makes them less willing to commit risky crimes using a security exploit. Or you could implement greater checks and balances to prevent financial fraud, which decreases the incentives for exploits. But these are society-wide changes that would not be under the control of the software manufacturer. (I'm not sure if there's anything a software company could do by themselves to lower the black-market value of a vulnerability in their product, other than voluntarily decreasing their own market share so that there are fewer computers that can be compromised using their software! Can you think of any other way?)
Raising the infinite bug threshold for the product, on the other hand, may require re-writing the software from scratch, or at least the most vulnerable components, paying stricter attention to security-conscious programming standards. Professor Kohno said after his talk that he believed that if the programmers of the car's embedded systems had followed better security coding practices, such as the principle of least privilege, then his team would not have found vulnerabilities so easily.
I still believe that cash prizes have the potential to achieve security utopia, at least with regard to the particular programs the prizes are offered for — but only where the "infinite bug threshold > black-market value" inequality holds, and only if the company is willing to offer the prizes. If the software is written in a security-conscious manner such that the infinite bug threshold is likely to be higher than the black-market value, and the manufacturer offers a vulnerability prize at least equal to the black-market value, then virtually all vulnerabilities which can be found for less than that much effort, will be reported to the manufacturer and fixed. Once that nirvana has been achieved, for an attacker to find a new exploit, the attacker would have to be (1) irrational (spending an estimated $70,000 to find a vulnerability that is only worth $50,000), and (2) evil beyond merely profit motive (using the bug for $50,000 of ill-gotten gain, instead of simply turning it in to the manufacturer for the same amount of money!). That's not logically impossible, but we would expect it to be rare.
On the other hand, for programs and classes of vulnerabilities where "infinite bug threshold < black-market value", there is literally nothing that can be done to make them secure against an attacker who has time to find the next exploit. You can have multiple lines of defense, like installing anti-virus software on your PC in case a website uses a vulnerability in Internet Explorer to try and infect your computer with a virus. But Kaspersky doesn't make anything for cars.
Bennett Bennett Bennett Bennett Bennett Bennett! I Bennett love Bennett Bennett! Nothing Bennett is Bennett more Bennett entertaining Bennett than Bennett reading Bennett what Bennett a Bennett person Bennett which Bennett I Bennett have Bennett no Bennett idea Bennett who Bennett they Bennett are Bennett has Bennett to Bennett say!
Why do words, suddenly appear
Every time, Bennett's here?
Just like me
you long to be
free from this
" I was an early advocate of companies offering cash prizes to researchers who found security holes in their products, so that the vulnerabilities can be fixed before the bad guys exploited them. I still believe that prize programs can make a product safer under certain conditions. But I had naively overlooked that under an alternate set of assumptions, you might find that not only do cash prizes not make the product any safer, but that nothing makes the product any safer — you might as well not bother fixing certain security holes at all, whether they were found through a prize program or not."
Is the whole premise of this article Bennett having a conversation with himself, talking about his previous points that no one also cared about? I understand slashdot is trying to start doing op-eds by having this guy write. But everything he writes is this long-winded, blowhard, arrogant, ego-massaging nonsense that no one but him cares about. Here he's writing about his previous writings and how his thoughts have changed..in a poorly-written article with no sense of a conclusion..just rambling.
Bennett is also not an information security expert ..he's just a blowhard..can we have someone really involved in information security, like Bruce Schneier, write articles for Slashdot instead of this nonsense?
You should be ashamed of your apathy.
Which has more power: the hammer, or the anvil?
Or it's like pretty much anything bad. Just because you cannot eliminate it completely doesn't mean you give up fighting. Murder is bad, and no matter how many we arrest, there will always be more murderers. That doesn't mean we should eliminate the police.
This "article" might actually be the dumbest thing Bennett Haselton has ever written, which puts it in legitimate contention for dumbest thing anyone has ever written.
Paying people to find bugs and report them responsibly does give those people an incentive to not do something worse with them.
In a way, this economy takes possible would-be black hats and turns them into white hats. I suspect there are far fewer people capable of finding every last exploit than there are exploits, so if we keep those people busy and paid doing what they do best, at least they won't be doing something more nefarious.
People talk about bug free code. It is a matter of won't, not a matter of can't.
Sometimes, there are products out there which can be considered "finished". Done as in no extra features needed, and there are no bugs to be found. Simple utilities like /usr/bin/yes come to mind. More complex utilities can be honed to a reasonable degree of functionality (busybox comes to mind.)
The problem isn't the fact that secure or bug free software can't be made. It is that the procedures and processes to do this require resources, and most of the computer industry runs on the "it builds, ship it!" motto [1]. Unfortunately, with how the industry works, if a firm does do the policy of "we will ship it when we are ready", a competitor releasing an early beta of a similar utility will win the race/contracts. So, it is a race to the bottom.
[1]: The exception to this rule being malware, which is probably the most bug-free code written anywhere these days. It is lean, robust, does what it is purposed to do, and is constantly updated without a fuss.
There is no such thing as a "black market value" of a security vulnerability. Both the demand and supply have curves. I.E there are security researchers who would demand say 1 million bucks before selling the bug to the CIA (because they view that action as unethical, illegal and risky careerwise) while they would gladly accept 10.000$ in a responsible disclosure offer. Other color hats would go to the highest bidder. Similarly, there are large transaction costs and information asymmetries, it's not necessarily true that the demand and supply meet or that they can trust each other. A spy agency might rather develop in house (at a much larger cost) then shop around and rise suspicion.
In short, offering a non-trivial sum of money will always increase the costs of the average attacker and might completely shut off the low impact attacks like spam zombification, email harvesting etc., the developers of which can't invest millions in an exploit but would gladly use the free zero day+exploit just made public.
They're like the small kitchen cockroaches in suburbia. You never can get rid of them, so all you can do is mitigate periodically
I don't like Bennett Haselton's posts either, but isn't that a bit harsh?
It's retarded to assume that you can't make a product secure; there aren't "infinite" bugs, there are obviously a finite number of bugs in any piece of software, and anyone who thinks otherwise either has some strange mental illness or doesn't understand software. But the reason bug bounties mostly don't work has nothing to do with the author's wiffle-waffle, it's just simple math and cost of labor. If I have the level of skills required to find security holes in a large piece of software like Windows or IE, chances are I can sell my labor at a minimum of $50/hour. To find a bug, I'm likely going to have to spend several days or weeks at it. If there's a $1000 bounty, that means I can spend at most 20 hours on the problem until I am literally losing money in opportunity costs. And hackers have to pay their mortgages and bills too. It's kind of insulting to think that an experienced security expert is going to labor away to find your bugs for you at well below market rates for that work if you had to pay someone to do it, as if a dog getting excited about a small treat, it's patronizing and insulting. If it would cost a company like MS $100,000/annum to have a security expert on their dev team to find those same bugs, then any 'bounty' has to START at well above those effective hourly rates.
My other UID is three digits.
I did read far enough to realize that this person is an idiot. We need only look at the heartbleed bug. If a bounty was offered and resulted in a fix earlier the number of stolen keys would be less, but that is almost besides the point. Once closed they might find another bug, but the likelihood that it will leak private keys is extremely low. To use a car analogy, every car has problems. This is essentially like claiming that fixing the exploding gas tanks in a Pinto is of no use, because the car will still have other issues. Seriously?
Guns don't kill people; Physics kills people! - John Lithgow as Dick Solomon on Third Rock From The Sun
...the notion that if you can't make software bug free, you may as well not bother is just stupid on a scale that's hard to comprehend. I skimmed as much of that article I could stomach, but I'm done.
If we can't make cars crash proof, we may as well not make them safer.
If we can't make people immortal, we may as well stop advancing medicine.
You know what? If you can't find perfect stories, you may as well stop posting junk like this.
But you don't understand. Bennett discovered DIMINISHING RETURNS.
People need to know.
NEED. To. Know.
Why not a security compiler? Seems some clever, creative hackers could work up something which would take raw code, subject it to some scrutiny and give output/feedback. Perhaps even a security switch to the standard compilers or even a security test suite. Shouldn't be that hard to do.
Everything in the Universe sucks: It's the law!
People talk about bug free code. It is a matter of won't, not a matter of can't.
Sometimes, there are products out there which can be considered "finished". Done as in no extra features needed, and there are no bugs to be found. Simple utilities like /usr/bin/yes come to mind. More complex utilities can be honed to a reasonable degree of functionality (busybox comes to mind.)
The problem isn't the fact that secure or bug free software can't be made. It is that the procedures and processes to do this require resources, and most of the computer industry runs on the "it builds, ship it!" motto [1]. Unfortunately, with how the industry works, if a firm does do the policy of "we will ship it when we are ready", a competitor releasing an early beta of a similar utility will win the race/contracts. So, it is a race to the bottom.
[1]: The exception to this rule being malware, which is probably the most bug-free code written anywhere these days. It is lean, robust, does what it is purposed to do, and is constantly updated without a fuss.
Once upon a time, I read somewhere (Yourdon, possibly) that the number of bugs in a software product tends to remain constant once the product has reached stability. The number for IBM's OS/MVS mainframe operating system was somewhere in the vicinity of 10,000!
It's been likened to pressing on a balloon where when you squeeze one bump in, another pops out, because the process of fixing bugs itself introduces new bugs.
And OS/MVS is about the most critical software you could put on a mainframe. You can't just Ctrl-Alt-Delete a System/370. Or power it off and back on again. Mainframes are expensive, and expected to work virtually continually. Mainframe developers were expensive as well, since after a million dollars or so of hardware and software, paying programmers handsome salaries wasn't as big an issue back then. Plus there was no offshore race to the bottom where price trumped quality at the time. In fact, there wasn't even "perma-temping" yet.
Still, with all those resources on such an important product, they could only hold the bug count constant, not drive it down to zero.
Actually speaking of OS/MVS, there's a program (IEFBR14) whose sole purpose in life is to do nothing. There have been about 6 versions of this program so far, and several of them were bug fixes. More recently, it had to be upgraded to work properly on 64-bit architecture, but some of the bugs were hardware-independent.
First, bugs in a given program are not infinite in number. By definition. Because the code itself is finite. Finite code cannot have infinite bugs. Also, due to the nature of code and how it is created, patching one bug usually also takes care of many others. If you have a buffer overflow problem in your input routine, you need only patch it once, in the routine. Not everywhere that routine is being called.
I have spent a few years (closer to decades now) in IT security with a strong focus on code security. In my experience, the effort necessary to find bugs is not linear. Unless the code changes, bug hunting becomes increasingly time consuming. It would be interesting to actually do an analysis of it in depth, but from a gut feeling I would say it's closer to a logarithmic curve. You find a lot of security issues early in development (you have a lot of quick wins easily), issues that can easily even be found in a static analysis (like the mentioned overflow bugs, like unsanitized SQL input and the like), whereas it takes increasingly more time to hunt down elusive security bugs that rely on timing issues or race conditions, especially when interacting with specific other software.
Following this I cannot agree that you cannot "buy away" your bug problems. A sensible approach (ok, I call it sensible 'cause it's mine) is to get the static/easy bugs done in house (good devs can and will actually avoid them altogether), then hire a security analyst or two and THEN offer bug hunting rewards. You will usually only get a few to deal with before it gets quiet.
Exploiting bugs follow the same rules that the rest of the market follows: Finding the bug and developing an exploit for it has to be cheaper than what you hope to reap from exploiting it. If you now offer a reward that's level with the expected gain (adjusted by considerations like the legality of reporting vs. using it and the fact that you needn't actually develop the exploit), you will find someone to squeal. Because there's one more thing working in your favor: Only the first one to squeal gets the money, and unless you know about a bug that I don't know about, chances are that I have a patch done and rolled out before you got your exploit deployed. Your interest to tell me is proportional to how quickly I react to knowing about it. Because the smaller I can make the window in which you can use the bug, the smaller your window gets to make money with the exploit, and the more interesting my offer to pay you to report the bug gets.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
The notion that you can't have code without these flaws (buffer overruns, dangling pointers, etc) is just asinine. I've worked on significant codebases without any such flaws. You just have to adopt a programming style that doesn't rely on being mistake-free to avoid the issues.
Want to end the danger of buffer overruns? Stop using types where it's even possible.
Want to end the danger of dangling pointers? Managed code doesn't do anything to solve this problem, and is often the worst offender since coders often stop thinking about how memory is recycled, and well-formed objects can hang around in memory for quite some time waiting on the garbage man. So you have to write code where every time you use an object you check that it hasn't been freed, and importantly hasn't been freed and then re-used for the same object! (That happens on purpose in appliance code, where slab allocation is common.)
Heck, for embedded code I simply wouldn't use dynamic allocation at all. All objects created at boot, nothing malloced, nothing freed. Everything fixed sized and only written to with macros that ensure no overruns. I wrote code that way for 5 years - we didn't even use a stack, which is just one more thing that can overflow. That style is too costly for most work, but it's possible, and for life-safety applications it's irresponsible to cheap out.
Socialism: a lie told by totalitarians and believed by fools.
Bennett, I think it's mostly that you simply DON'T REALLY HAVE ANYTHING THAT INTERESTING TO SAY.
What you've presented us with is a smugly self-congratulatory "inner dialogue" in which you discovered that "at some point, it becomes more expensive to find bugs than the bug bounty will compensate you!"
Anybody with more than a second grade education can reason this out for themselves, but you saw fit to drone on for pages about it. You do this a lot. it's not informative, interesting, or insightful. You should probably either: a) up your game and offer us some real thoughtful insight; or b) shut the fuck up and take it back to livejournal.
Congratulations, you've discovered the law of diminishing returns - you're a fucking legend.
Look where that got the electrical engineers.
http://news.slashdot.org/story...
char*f="char*f=%c%s%c;main(){printf(f,34,f,34);}";main(){printf(f,34,f,34);}
Aside from the obvious ethical reason, I see two reasons more important than the $1,000 to go "white hat" rather than "black hat".
When a potential employer Googles my name, I want them to find my name on CVEs, Github commits, etc. - demonstrable proof that I do in fact find and fix real-world issues. I'm working on that. Right now, I'd have to point out my contributions, they aren't easily found via Google. For that, having a company or other organization publicly acknowledge my work is much more valuable than $1,000, if it helps me land a great job.
On the other hand, selling it on the black market could put me in federal prison. If the god guys offer me $1,000 plus a reputation boost, while the bad guys offer me $5,000 plus a possible prison sentence, I think I'll take the good guys' offer. That $1,000 could, in some cases, be enough to pay someone's past-due rent so they don't feel they have HAVE to capitalize on it in a bad way.
The other scenario I see is that several times per year I notify a smaller company of some security hole I noticed in passing. I haven't thoroughly probed it, just noticed "gee, it throws an error on O'Doole, it's probably not escaping the input and therefore vulnerable to SQL injection". Sometimes I don't bother to track down the proper person to notify and go notify them. Sometimes, I send an email to the only readily available email address, customer service, and the $8 drone on the other end replies with a form letter wholly inappropriate to the situation, so they obviously don't understand what I told them. In those cases, I'll likely not spend much time trying to find another person at the company. If most companies paid even $100 for a bug bounty, that would make it worth my time to spend a few minutes finding their report form and use it. Heck, at $100 per SQL injection vulnerability I could make a good living finding and reporting those for six hours per day.
An interesting argument is one that proceeds from premises and reasoning steps that individually are uncontroversial, but taken together, lead to a conclusion that is far from obvious, or even seems wildly counterintuitive -- but which, if you accept the premises and reasoning steps, you have to accept the conclusion as well. The more counterintuitive the conclusion, the more interesting the argument, as long as the premises and reasoning steps are sound. Even if you disagree with the conclusion, the interesting part is to try and identify the premise or reasoning step that you disagree with.
The problem is that many people respond to these arguments simply based on how they "feel" about the conclusion, and that's missing the point.
The problem is that the premises seem really dubious, and since there is no actual agreement about bug bounties that I've noticed, the conclusion is not particularly surprising.
"When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes