Slashdot Mirror
Study Finds Bug Bounty Programs Extremely Cost-Effective
itwbennett writes "U.C. Berkeley researchers have determined that crowdsourcing bug-finding is a far better investment than hiring employees to do the job. Here's the math: Over the last three years, Google has paid $580,000 and Mozilla has paid $570,000 for bugs found in their Chrome and Firefox browsers — and hundreds of vulnerabilities have been fixed. Compare that to the average annual cost of a single North American developer (about $100,000, plus 50% overhead), 'we see that the cost of either of these VRPs (vulnerability reward programs) is comparable to the cost of just one member of the browser security team,' the researchers wrote (PDF). And the crowdsourcing also uncovered more bugs than a single full-time developer could find."
The major problem is that on-staff developers are usually discouraged from going on bug-hunts. Management would rather have them developing new features, so they won't allocate time towards finding bugs. When what the company policy towards finding bugs is conflicts with how your manager assigns you tasks, guess which one wins. Worse, most of the time an employee who ignores his to-do list to go find problems ends up penalized either explicitly (by bad reviews) or implicitly (negative impact from people being annoyed that he made work for them). Outsiders in these bounty programs don't have to worry about a manager assigning them 100% to new features and 0% to finding vulnerabilities and they don't have to worry about the impact of bad reviews or negative comments by managers about the extra work they created for everybody.
Mostly shows how being good at finding bugs is a different skill than being good at job interviews.
isn't $570,000 / $150,000 about 3.8 people? (articles numbers.) Still probably a good deal, but not quite as good.
http://dilbert.com/strips/comic/1995-11-13/
This is indeed true specially for popular companies with rather mature SecOps that pay minimum wages for vulnerabilities that are indeed hard to find or require a pretty darn good skill level to discover. Some of them even only offer swag in exchange of finding serious threats such as persistent XSS or authentication bypass. They maybe feature the researcher in some blog post to publicly thank him and attract the wannabe crowds.
Having said that, I myself have participated in several of these programs (with varying success) and come to realize that probably Google and Facebook are the only VRPs currently paying reasonable wages for bugs in terms of cost efficiency for the researcher.
On the other hand, some of us just enjoy from time to time trying to find security bugs for fun (maybe because we are huge nerds) so these programs offer a great opportunity to test things and not risking ending up in jail.
---------
Thinking never hurt anybody --MacGyver
as a corporation is abdicates you from the responsibility of things like health insurance in countries like america that have very expensive coverage individuals typically cannot afford. In more advanced countries like sweden or canada, youre indirectly allowing a government to subsidize a component of your under-the-table employment of coders and hackers. expenses like retirement, life insurance, dental coverage and the cost of work-related activities like ice cream socials are then realized as a savings. In my opinion coders and hackers must be very careful when engaging in bug bounty as the cost of a programmer including benefits is often not fully reimbursed when they find and patch a bug. even if that is not a primary consideration, the ethics of fixing googles problems are worth considering
small projects like mozilla should get to do it, as theyve consistently demonstrated a moral and ethical commitment to protecting the internet for all humankind. Google, a major multinational corporation that lobbies congress for H1B legislation, is in a bit more of a grey area. Chrome is an offering in which its user becomes the product, the final objective to sell the subjects data to various other corporations and earn a profit.
Good people go to bed earlier.
Maybe they should have compared the salary of a QA person instead of a developer. As a developer, I find lots of bugs, and then fix them. I also fix the bugs that QA finds, but usually spend a lot of time trying to figure out how to reproduce the issue ("uhh, first I clicked on this and then I clicked on that and then something weird happened").
Anywhile, it's hard to crowd source a product that has not been released yet and most companies don't have the fan-bois and gurls to even consider this strategy.
Browsers have very large installed base. There are enough bug spotters even if a very small fraction of them actually hunt and report bugs. Even then, the bounty is for finding the bugs, not fixing the bugs that includes the cost of coming up with a fix, verifying it fixes the problem, testing to make sure it does not create new problems and rolling out the fix.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
That means that there is a strong incentive for companies to create insecure, crappy software and then let so-called "white hat hackers" fix their bugs at a discount. And because any other form of disclosure is illegal, the companies are pretty well protected from negative consequences of their bugs and deflect from their own negligence by blaming "black hat hackers".
Given that disclosure is also at the terms of the payer, you also get less transparency versus independent disclosure.
Twitter supports and protects racists - by smearing their critics with the "Hate Speech" label.
Because the sort of programmer that's good at finding/fixing these bugs...is not the sort of programmer that the interview process determines would be a "good fit" for the organization.
"Once we've identified and embraced our sickness, we'll have strength...and that's when we get dangerous." - John Waters
This is effective for the low-hanging fruit, i.e. the easy (relatively) to find security-related bugs. For things that require advanced techniques or expensive tools (like Fortify), it fails. Unfortunately, the harder to find bugs are still well within reach of spy agencies of all kind, including a number that is allowed to do industrial espionage (like the US or France).
So while this looks good on the surface, it is really just making the problem worse. The only exception is software that has very low security needs.
For reliability, it is about as ineffective, as only easy to identify bugs will be tracked down.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
I would hope Google is smart enough to know that you don't need an experienced developer to find bugs in their code. Aspiring developers fresh out of college are more then adequate. At 50k a pop google could have hired 7 PFYs spending 14,000 hours scouring code, hell give them 1k bonuses for each bug to keep them motivated.
Knowledge = Power
P= W/t
t=Money
Money = Work/Knowledge so the less you know the more you make
What I'm really shocked about is that you need a university to figure this out. Or rather do research on this. Companies figured this out quite some time ago and anyone with a functioning brain can see why. :)
What I'm more interested in is that king of people spend their time in participating in programs like this. The chances that you find a bug are not that big. The financial reward, given the amount of time you will spend on finding a bug is probably also relatively small.
From a company's point of view on the other hand, it's great. Many people working for you. For free. A job well done
Privacy is terrorism.
or one instead offers ``certificates of deposit'' in the (fictional) ``Bank of San Seriffe'': http://en.wikipedia.org/wiki/Knuth_reward_check
William
(who is quite bummed that he didn't get his reward check back when Dr. Knuth was using Wells Fargo as his bank: http://www.truetex.com/knuthchk.htm )
Sphinx of black quartz, judge my vow.
For a lot of these people, it might be a hobby. If it weren't for bug hunting for a bounty, they might be working on open source software instead with no payout at all. For those people, the payout is infinitely greater even if it amounts to $2.50/hr. Most people are just happy to have a hobby that breaks even, nevermind nets a profit.
Many of the people working on these things will also have full time jobs as security researchers. The extra financial incentive for a bug just means that they'll be applying their bug-finding technique to your codebase instead of to someone else's.
I am TheRaven on Soylent News
Is good to reward people that find security holes, at the very least because is a safer bet than selling them in the black market, or keeping them for yourself or the government to exploit them. But it should not be a replacement for actually having dedicated people activelly working for your security that will report to you if something weird is there, some could actually go to the black market (or be found by government teams and never disclosed that it is there because is an useful cyberweapon) and you must be proactive from your side
It sure isn't the average in Canada.
It's not surprising at all that piecemeal work, with no provision for healthcare, vacation etc. - much less reliable, ongoing income - is more profitable for business.
Why should technology workers be intrigued or inspired by this? Why is this information presented to technology workers as another avenue to praise Google's or Mozilla's cleverness? And why do technology workers so consistently dig their own graves by latching onto this kind of ideology and failing to fight for labor rights?
I believe it was a minvan
where you have millions of folks looking at your free software for long periods of time. If you're a commercial software vendor, however, with a $10,000 non web-based package and at most a few thousand users (There are still a *lot* of these), then this approach is very unlikely to succeed. Commercial software users are rarely interested enough to report a bug that doesn't actively interfere with their daily work.
Please do not read this sig. Thank you.
I get paid to audit code, so I'm biased.
The article says that no one employee could find hundreds of bugs and that's true. But when you hire employees you are building a process. Improving the process by writing a new QC script can eliminate hundreds of bugs over a couple years. These are not attributed to one employee and since the offending code is not committed then they aren't even counted as bug fixes.
Offering a bug bounty, on the other hand, is a unpredictable thing and you'll get random fixes. It is valuable because it provides a fresh perspective.
My guess is that if you collect a few bug bounties then Google will send you a recruiting email. It might be more expensive to hire you to work full time it's still a worthwhile thing.
developers don't "create" bugs, we don't sit down and say, hey, lets create a bug! and then go about making one, most of the time, we believe our code doesn't have bugs, cause if we thought it had bugs, we'd write it a different way, or we'd know about the bug in the first place and it'd be in our list of things to fix, normally those things are fixed quickly because we knew it was there, but things we don't know about, well, how do you expect us to find it? I didn't find it whilst I was writing the code and I'm the brain doing the typing, if I can't find it then, what do you think the chance of me finding out afterwards will be?
other people are very good at finding bugs in my code, if they exist, because they have a different mindset and think about things differently than I do, they think of a circumstance I didn't think of, great, you found the bug! but I didnt know it was there.....
so I think it's kind of normal that other people will find bugs in your code that you didn't know existed, so it makes perfect sense to reward those people to find them, paying ME to find them is going to not pay off in a big way, cause if I knew how to find bugs in my own code, I would have done it already and fixed it.
So UC Berkeley should compare the number of bugs found by researchers vs the number that Google's Internal QA Dept has found.
Then we'll know if it was really worth while. Since Google would never publish the number of bugs they find internally, all this data is worthless.
It is nice though getting people to QA your projects for basically free.
on the type of bug. If it's security related, you can never be sure that the customer that finds it will report it instead of exploit it.
And you don't have to worry about employment laws.
get outsourced to india or cut entirely?
---Saying gnome 3 is better than windows 8 not so much a compliment as it is damning with light praise.