Study Finds Bug Bounty Programs Extremely Cost-Effective

itwbennett writes "U.C. Berkeley researchers have determined that crowdsourcing bug-finding is a far better investment than hiring employees to do the job. Here's the math: Over the last three years, Google has paid $580,000 and Mozilla has paid $570,000 for bugs found in their Chrome and Firefox browsers — and hundreds of vulnerabilities have been fixed. Compare that to the average annual cost of a single North American developer (about $100,000, plus 50% overhead), 'we see that the cost of either of these VRPs (vulnerability reward programs) is comparable to the cost of just one member of the browser security team,' the researchers wrote (PDF). And the crowdsourcing also uncovered more bugs than a single full-time developer could find."

Comments

Incentives

[Todd Knarr]

The major problem is that on-staff developers are usually discouraged from going on bug-hunts. Management would rather have them developing new features, so they won't allocate time towards finding bugs. When what the company policy towards finding bugs is conflicts with how your manager assigns you tasks, guess which one wins. Worse, most of the time an employee who ignores his to-do list to go find problems ends up penalized either explicitly (by bad reviews) or implicitly (negative impact from people being annoyed that he made work for them). Outsiders in these bounty programs don't have to worry about a manager assigning them 100% to new features and 0% to finding vulnerabilities and they don't have to worry about the impact of bad reviews or negative comments by managers about the extra work they created for everybody.

Cost of fiunding bugs != cost of fixing them.

[140Mandak262Jamuna]

Browsers have very large installed base. There are enough bug spotters even if a very small fraction of them actually hunt and report bugs. Even then, the bounty is for finding the bugs, not fixing the bugs that includes the cost of coming up with a fix, verifying it fixes the problem, testing to make sure it does not create new problems and rolling out the fix.