Slashdot Mirror


Companies Are Paying Millions For White Hat Hacking (nypost.com)

White hat hackers "are in very high demand," says PwC's director of cyber investigation and breach response, in a New York Post article titled "Companies are paying millions to get hacked -- on purpose." An anonymous reader quotes their report: HackerOne, a San Francisco-based "vulnerability coordination and bug bounty platform," reports that it has some 800 corporate customers who paid out more than $15 million in bonuses to white-hat hackers since its founding in 2012. Most of that bounty was paid in the past two years, as companies have become more aware of their cyber vulnerabilities. Clients that have used the platform include General Motors, Uber, Twitter, Starbucks and even the US Department of Defense.
Google paid $3 million last year through its own bounty program, according to HackerOne's CEO Marten Micko, who touts his company's "turn-key" solution -- a platform which now offers the services of 100,000 ethical (and vetted) hackers. "With a diverse group, all types of vulnerabilities can be found," Micko told TechRepublic. "This is a corollary to the 'given enough eyeballs' wisdom... they find them faster than other solutions, the hunting is ongoing and not happening at just one time, and the cost is a tenth of what it would be with other methods." And one of the platform's white hat hackers has already earned over $600,000 in just two years.

58 comments

  1. Payouts are garbage, though by Anonymous Coward · · Score: 1

    10k to fix a bug that could destroy everything. The black market is still better.

    1. Re:Payouts are garbage, though by ls671 · · Score: 1

      It has always been the same, technology or not; people with skills that use them according to their values. Send me the job offers, I might consider them...

      --
      Everything I write is lies, read between the lines.
    2. Re: Payouts are garbage, though by muffen · · Score: 1

      What exactly is the story here? Companies, all of them put together, are paying millions for everything, I'm pretty sure the water bill from flushing after taking dumps is in the millions, and it wouldn't surprise me if Google spends $3 million on food every month.

    3. Re: Payouts are garbage, though by Anonymous Coward · · Score: 0

      And you get a free black hoodie!

    4. Re: Payouts are garbage, though by Anonymous Coward · · Score: 1

      Admittedly, the only real news here is that this is an above board platform to connect such talent to organizations seeking such, as opposed to networking over IRC etc...

      You've got to admit that middlemen successfully making a buck usually signifies something or another... back in the day before guys like the L0pht crew you were more likely to end up under threat of arrest or sued than paid as a consultant.

    5. Re: Payouts are garbage, though by Anonymous Coward · · Score: 0

      Yep. This is why I haven't bothered trying. They need to seriously up the bounties.

    6. Re:Payouts are garbage, though by phantomfive · · Score: 1

      The black market will always pay better. If companies increase their offers, then the black market will increase them even higher. Although as the prices rise, the number of buyers on the black market will decrease.

      --
      "First they came for the slanderers and i said nothing."
    7. Re:Payouts are garbage, though by martenmickos · · Score: 1

      Given the ease of submission and speed of payment, a bug bounty can be very well worth it. On HackerOne, there is a hacker who made over $600,000 in two years with most of the individual bounties well under $10k.

    8. Re: Payouts are garbage, though by martenmickos · · Score: 1

      What I find interesting is that a regular newspaper will write about this despite it being a highly technical topic. The readers of New York Post are regular citizens. This shows that software security and the hunt for bugs are becoming important enough to be presented to the broader public.

    9. Re:Payouts are garbage, though by martenmickos · · Score: 1

      This is an interesting question. We don't really know what will happen long term. One possibility, as you point out, is that black markets will always outpay any other market. Another possibility is that the ethical hacker community will become so large and strong that they will find all those same vulnerabilities and deliver them to the system owners before the black market gets to build exploits and use them for nefarious purposes. It takes just one ethical hacker who finds a critical 0day to deliver it to a service like HackerOne, and the market for that vuln is over. Although asymmetry is usually in the favor of the criminal actor, in this case it is in the favor of ethical behavior. One ethical hacker can put an end to the sale of a 0day on the black market.

    10. Re:Payouts are garbage, though by phantomfive · · Score: 1

      That won't happen. A company that writes security vulnerabilities will continue to write them, thus providing an endless supply. Until something changes within the company, there will be a bountiful harvest for both black and white hat.

      --
      "First they came for the slanderers and i said nothing."
    11. Re:Payouts are garbage, though by gweihir · · Score: 1

      Indeed. This is completely bogus. People doing it will go for the low-hanging fruit and if they find something really juicy by accident, they can easily make one order of magnitude more money on it. Nothing more complicated will ever get reported to the company. This may also explain why the cost of this is 1/10 of other methods: It has far less than 1/10 of the results and is dangerous in addition.

      Now, a really competent security review will be expensive, but it will look at things like code quality, design and architecture, competence of the implementers and maintainers, processes, roadmap, etc. It will, for example, find cases where only one safeguard is effective (and hence things cannot get hacked), but the risk is hight, because you usually want two independent safeguards. It will find things were technology is not quite there yet to attack them. It will find conditions that were unknown, but must be met in order for a system to remain secure.

      Of course, if done by one of the big IT consulting agencies, it will only pretend to do all these things, but as compensation it will be even more expensive. The amount and quality of fail I have seen in reports of IT evaluations from big names is staggering.

      In short, this is the moronic version of IT security, which is not worth the money saved on it, even if it would cost 1/100 of a real security review. It may have some short-term benefits for the bonuses of those having made this utterly stupid decision, but that is it. Long-term, it is disastrous.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  2. Millions is not always a lot by Pirulo · · Score: 1

    A $3M expense from Google is not a considerable sum. They are cheaping out by only expending that little when you consider all data they store and are pseudo-responsible for.

    1. Re: Millions is not always a lot by Anonymous Coward · · Score: 0

      Noone seems to be "responsible". That is kinda part of the problem.

  3. Not as good as it sounds by Anonymous Coward · · Score: 0

    So the world's best (or at least, best-paid) white-hat makes $150k/year? Unless they're just doing it in their free time, they could make more than that working a standard corporate security job in a good area, or they could make 5-10 times as much as a black-hat.

    1. Re: Not as good as it sounds by Anonymous Coward · · Score: 0

      A lot of people aren't suited to corporate environments, no matter how 'liberal'. Some of the most skilled troubleshooters I've known had non tech day jobs.

    2. Re: Not as good as it sounds by Anonymous Coward · · Score: 0

      What's wrong, did a bad outside pentester roughly caress your private servers?

      It's just a security centric services platform for consultants/contractors.

    3. Re:Not as good as it sounds by gweihir · · Score: 1

      Very much so. And you can make a lot more as a gray-hat, with no risk of prison time. These people are basically a bit more advanced amateurs with big egos that exploit themselves.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  4. Ten years too late... by __aaclcg7560 · · Score: 0

    I was a black box tester for nearly seven years. When I graduated from community college with A.S. degree in computer programming in 2007, I wanted to get a job as a white box tester. Never got hired. Went into I.T. support and the rest was history.

    1. Re:Ten years too late... by Anonymous Coward · · Score: 0

      Even with your 800 headhunters that you talked to 10 times a day for two years? Seems to me you are extremely inefficient in your "energy expended" to results ratio.

    2. Re:Ten years too late... by __aaclcg7560 · · Score: 0

      Even with your 800 headhunters that you talked to 10 times a day for two years?

      You need to work harder at misrepresenting my positions. How do you ever expect to get ahead in life as a Troll?

      Seems to me you are extremely inefficient in your "energy expended" to results ratio.

      That's because people are involved. If this was rocket science, 92 million Americans would have coal miner jobs.

    3. Re:Ten years too late... by Anonymous Coward · · Score: 0

      Lying is lying dude. Trump does it. You do it. Your fat delusional reality is just one look in the mirror away.

    4. Re:Ten years too late... by __aaclcg7560 · · Score: 1

      Lying is lying dude. Trump does it.

      Of course. He's a politician.

      You do it.

      I'm not a politician.

      Your fat delusional reality is just one look in the mirror away.

      I shave each morning. So what?

    5. Re:Ten years too late... by Anonymous Coward · · Score: 0

      you're not qualified to comment on getting ahead. literally everything about you is below average but your weight. and you are proud of being our clown. good for you.

    6. Re:Ten years too late... by __aaclcg7560 · · Score: 1

      you're not qualified to comment on getting ahead. literally everything about you is below average but your weight. and you are proud of being our clown. good for you.

      You sound like my mother. Good thing I stopped listening to her when I was teenager. Otherwise, I would have committed suicide and the world would be worse off place than it is now.

    7. Re:Ten years too late... by Anonymous Coward · · Score: 0

      No one gives a fuck who you listen to or don't. You're just a clown here to entertain us. A fat mediocre fuck who is happy with his life by thinking big about small things. And likely spoonfuls of prozac in the mornings and most definitely the anti-depressant of a 4k calorie diet.

      You're a janitor who's proud of his job. The world would also be a worse off place without janitors, and their job is hard. A way you deal with accomplishing nothing and not having the capacity to do so is by talking down to people who do. You even make up this virtual world for yourself where you are better than everyone. Where you for example sit at work "waiting for a script" so you're on slashdot. Do you actually believe that? How's that script of yours doing? I believe you are a fat loser with no brains, so this is where you socialize. Bland, Fat, and Boring as shit.

      I'm here to shit all over people. I get pleasure and relaxation from putting down fat ugly losers. You don't need to listen to me. It's a website, and I'm not giving you advice. I am enjoying myself at your expense. As do most people.

    8. Re:Ten years too late... by __aaclcg7560 · · Score: 1

      And likely spoonfuls of prozac in the mornings and most definitely the anti-depressant of a 4k calorie diet.

      I don't take anti-depressants. Never had, never will. My current diet is 1,500 calories per day.

      Where you for example sit at work "waiting for a script" so you're on slashdot. Do you actually believe that? How's that script of yours doing?

      I just finished re-writing the parser section of my Python script. It's currently grabbing, parsing and saving my 8,000+ comment history into a CSV file. This usually takes 30 minutes.

      I believe you are a fat loser with no brains, so this is where you socialize.

      I was at the Silicon Valley Comic Con 2017 this weekend, where I posted some comments on Slashdot in between events. Check out my William Shatner video from the sixth row at the City National Civic.

      SVCC 2017 - William Shatner - The Bicycle
      https://www.youtube.com/watch?v=ppJiwDSz0j8

    9. Re:Ten years too late... by Anonymous Coward · · Score: 0

      basically, you're full of shit. you always answer all the comments and fast. you claim it's while waiting for scripts to finish at your sr. systems admin job that requires 20 years of experience but pays the salary of a helpdesk monkey. I do believe the part where you have to save up 4 years for a shitty vacation the rest of us can take on a whim, and that you have a job the rest of us consider shitty. I don't believe you are sitting there waiting for scripts 7 days a week. Or maybe I do. Is you title sr computer operator?

      you claim you eat about the same as a skinny woman yet you are a fat fuck who has a chin on his chin, and another on the back of his neck, and I bet when someone gets stuck next to you on a plane they ask to be moved.

      The fact that you went to a comic book event to see captain kirk actually proves my point about you socializing on slashdot. You claiming otherwise is a monkey saying "I'm not a monkey - just look at my pictures at the zoo"

    10. Re:Ten years too late... by pnutjam · · Score: 1

      Do you mean "black hat", because when I read black box, I assume you are testing a device with no documentation. I'm not sure what a white box is, maybe well documented?

    11. Re:Ten years too late... by __aaclcg7560 · · Score: 1

      I don't believe you are sitting there waiting for scripts 7 days a week. Or maybe I do. Is you title sr computer operator?

      I have a regular job that pays the bill and I have my own company. It's not unusual for entrepreneurs to work seven days a week. I'm currently running a script pinging systems while listening in to a conference call at my bill-paying job.

      [...] you are a fat fuck who has a chin on his chin [...]

      That picture was taken four years ago and I didn't start my 1,500-calorie diet until recently. There two types of people on Slashdot: the ones who see my picture and come back with "you are a fat fuck", and everyone else.

      I bet when someone gets stuck next to you on a plane they ask to be moved.

      I never had that problem. I'm heavy but I'm not wide enough to pay for an extra seat.

      The fact that you went to a comic book event to see captain kirk actually proves my point about you socializing on slashdot.

      I also saw astronaut Buzz Aldrin's presentation on going to Mars.

      http://www.sfchronicle.com/science/article/Astronaut-Buzz-Aldrin-champions-science-during-11092087.php

    12. Re:Ten years too late... by __aaclcg7560 · · Score: 1

      Do you mean "black hat", because when I read black box, I assume you are testing a device with no documentation. I'm not sure what a white box is, maybe well documented?

      Black box testing is testing something without knowing how it works inside. White box testing is testing something while knowing how it works inside. Grey box testing is a mixture of the two, say, a device with an exposed API that doesn't document the internal workings.

    13. Re:Ten years too late... by pnutjam · · Score: 1

      OK, if nobody would hire you for "white box", why couldn't you continue "black box" testing with responsible disclosure. There is a lot of money in that.
      I was confused because you indicated an inability to break into the industry and assume you misspoke trying to indicate what you where doing was unsanctioned and not necessarily legal.

    14. Re:Ten years too late... by __aaclcg7560 · · Score: 1

      OK, if nobody would hire you for "white box", why couldn't you continue "black box" testing with responsible disclosure. There is a lot of money in that.

      My black box testing experience was six years as a video game tester and lead video game tester, six months as software tester for a virtual world and six week as a software tester for an ebook reader. Some hiring managers would find my experience "lacking" because none of it was a "mainstream" product. I didn't want to continue as a video game tester because all the local companies were moving to Southern California for the Hollywood convergence that never happened. Recruiters kept offering me positions in IT support when I started applying for white box testing positions.

      I was confused because you indicated an inability to break into the industry and assume you misspoke trying to indicate what you where doing was unsanctioned and not necessarily legal.

      Black/white hat hackers are different category.

    15. Re:Ten years too late... by Anonymous Coward · · Score: 0

      Blah blah blah fat 4 years ago, just started a diet blah blah irrelevant - you're a fat fuck, that's a fact. In your mind you just started a diet and you're no longer fat. The fact is, you look worse now than in that photo because you're now older. After 50 failed diets you're going to be even worse as your metabolism gets worse and worse with age. Otherwise you wouldn't be fat now. Which you are.

      Seeing a presentation about space is socializing? Gotcha. You know what I did? I went out to a restaurant then hung out with a bunch of girls at a k-pop karaoke. Today I decided to go to Italy for euro labor day for a week or two. Went online, booked the trip - gonna work remote for a bit. After 20 years of experience, I can do that without saving and planning, as it's going to cost just a few day's wages. Not quite a show by some unknown stripper in the evening gymnasts in vegas though. I'd need to save up for that. Till about lunch.

      So we got a lardass college flunky whose parents are disappointed by him, who in his middle age sits on slashdot all day, and goes to uggo nerd-fests to socialize on the weekends. You have said literally not a single thing to contest this.

      And how did this thread start? With the fact that no one would hire your unqualified fat ass for even something as basic as software testing. You were only allowed to do functional testing. Which is what 14 year old dorks do. It's called playing video games.

      You say this news is 10 years too late. You would have never qualified at any point. You are too stupid.

    16. Re:Ten years too late... by __aaclcg7560 · · Score: 1

      Seeing a presentation about space is socializing? Gotcha. You know what I did? I went out to a restaurant then hung out with a bunch of girls at a k-pop karaoke. Today I decided to go to Italy for euro labor day for a week or two. Went online, booked the trip - gonna work remote for a bit. After 20 years of experience, I can do that without saving and planning, as it's going to cost just a few day's wages. Not quite a show by some unknown stripper in the evening gymnasts in vegas though. I'd need to save up for that. Till about lunch.

      You're the asshat with the drinking and financial problems! No wonder you sound so bitter in your replies. You should really lay off drinking $3,000 per night on wine. It's not healthy lifestyle.

    17. Re:Ten years too late... by gweihir · · Score: 1

      white box = you have all documentation, accounts, technical authorizations, and access to people
      gray box = you have some of the above, often in limited form
      black box = you know how to reach the target systems

      Black box pen-testing makes no sense, since it wastes a lot of time. The only reason to do it is that with a limited budget, it may not find things, i.e. create a false sense of security that management can then escalate as a great achievement.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
    18. Re:Ten years too late... by pnutjam · · Score: 1

      Thank you, that's why i was confused. Black hat make sense, black box, not so much.

    19. Re:Ten years too late... by gweihir · · Score: 1

      You are welcome.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  5. Is "white hat hacker" a Millennial/hipster term? by Anonymous Coward · · Score: 0

    Is "white hat hacker" a Millennial/hipster term? The described role sounds a lot like that of what we typically call a "network security analyst".

  6. Apparently more hackers need to learn .... by Anonymous Coward · · Score: 0

    More young hackers need to learn to take advantage of these types of programs, vs. trying to get paid or hired on their own by finding vulnerabilities and then threatening the businesses or institutions with exposing them if they aren't paid to provide a fix.

    This just happened at a community college in Maryland. One of the students started posting random flyers around the school, warning people that the network was insecure, and essentially trying to blackmail the college into hiring him (at least as a consultant) to fix the security problem.

    He did, indeed, find a vulnerability. Not surprising at all, since security has been a weak sport for the school for quite some time. (There's currently a lot of hiring and firing going on behind the scenes, related to trying to get more competent and security-conscious people working in I.T. there. Like many places, they suffered from long-tome employees at the top of the chain of command trying to cling to outdated technology as long as possible, and refusing to keep up with changes.)

    But his approach was NOT the right way to monetize his skill in finding the security flaw(s). The fact that the FBI escorted him off the campus last week attests to that.

    1. Re: Apparently more hackers need to learn .... by Anonymous Coward · · Score: 0

      Well, it's because you live in a police state. And apparently you love it since you keep asking for more of the same.

    2. Re: Apparently more hackers need to learn .... by Anonymous Coward · · Score: 0

      The OP lives in China, Russia, NK or any muslim country? (pick one, they are all backward and full of the death cult masquerading as a religion)

    3. Re: Apparently more hackers need to learn .... by Anonymous Coward · · Score: 0

      I'm eating pizza, sipping a lager, and watching Star Wars, all while trolling Slashdot and masturbating. Not bad for a police state!

    4. Re: Apparently more hackers need to learn .... by Anonymous Coward · · Score: 0

      What I find hilarious about your story is that even if he had already been a paid consultant for them they likely wouldn't have done shit about the issues, and he probably would've gotten the same treatment even without the flyers.

    5. Re: Apparently more hackers need to learn .... by Anonymous Coward · · Score: 0

      To be fair if you're doing it from inside a prison cell it still counts.

  7. Not the best, and it's more than $300,000 / year by raymorris · · Score: 1

    The summary says:
    --
      one of the platform's white hat hackers has already earned over $600,000 in just two years.
    --

    From that you got:
    > So the world's best (or at least, best-paid) white-hat makes $150k/year?

    Over $600K in two years is over $300K per year. No, that's not "the world's best paid white-hat". That seems to be how much one freelancer made from Hackerone - he or she may have made just as much from other avenues, and there is no reason to think this person is "the world's best-paid white hat - in fact there is good reason to think they are not.

    That is roughly the range of someone highly qualified, though - without pairing it with management or other fields such as writing. (Think Bruce Schneier - trained as a cryptologist, paid as an author and nerd-famous media personality).

  8. Re: Is "white hat hacker" a Millennial/hipster ter by Anonymous Coward · · Score: 0

    Marketing term, mainly by companies pushing pentest certificates and their associated flotsam.

  9. Vetting by Anonymous Coward · · Score: 0

    "the services of 100,000 ethical (and vetted) hackers."

    Something tells me the vetting wasn't very thorough.

    1. Re:Vetting by gweihir · · Score: 1

      At the cost-point they claim? They maybe have verified the email address for that ...

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  10. Re: Is "white hat hacker" a Millennial/hipster ter by Anonymous Coward · · Score: 0

    Also marketing from IT consulting companies trying to score security contracts with Fortune 500. It's working. A lot of corporate IT departments have taken up the "it's not if but when" mantra as gospel. Makes the current situation seem less fucked, but a lot of the higher ups will NEVER see it that way. Although, it's probably true for most companies with vendor support models. "No sir, there's no way this could have been prevented" will only work for so long.

  11. Re:Not the best, and it's more than $300,000 / yea by Anonymous Coward · · Score: 0

    Wow, one white hacker made over 600k! So we should just assume they all did!

    600k in bugs at extremely shitty payouts...I wonder how much money this guy would have gotten on the black market.

  12. I suppose the headline is true... by GameboyRMH · · Score: 1

    ...in the same sense that the NBA is paying millions for basketball-playing.

    --
    "When information is power, privacy is freedom" - Jah-Wren Ryel
  13. This is advertising post by Anonymous Coward · · Score: 0

    This isn't news. As said above $3 million is a joke for google to spend. That's not sounding like a big market. What this post actually is, is an advertisement for HackerOne and PwC's own new product for contract based security people.

  14. Re:Not the best, and it's more than $300,000 / yea by gweihir · · Score: 1

    The problem here is that this will not keep. There is definitely luck involved, and if this person was, say, working 80h weeks, then the compensation still sucks.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  15. Re:Is "white hat hacker" a Millennial/hipster term by gweihir · · Score: 1

    It can also be a "system security analyst" and a "software security analyst" or a lot more fuzzily an "IT Security Expert" or "IT Security Consultant". Of course, in a time where the BS term "Cyber" gets attached to anything, "Hacker" is actually a significant improvement.

    --
    Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  16. Not much luck over a two-year period by raymorris · · Score: 1

    My team and I do something similar periodically. Our experience is that luck is a short term phenomenon in the face of skill. Daniel may happen to find something pretty good in the morning, while I don't find much until the afternoon. Zach might find two interesting bits on Monday, none on Tuesday; Immad finds one on Monday and one on Tuesday. Over the course of a few days, our performance tends toward what you'd expect from our resume. Luck is very short term, skill is the controlling factor over the course of even days, and certainly over years.

    For example, guy who knows how to analyze a system to divide it into its components then focuses on the interactions between those components will find many more vulnerabilities than someone who focuses on one component and tries to find vulnerabilities within that component, internal to it. It's the interfaces between systems where most of the weaknesses are. Looking in the right places, the most likely places, is a skill, not luck.

    1. Re:Not much luck over a two-year period by gweihir · · Score: 1

      Luck is involved in the parts of finding things that others have not yet found and that hence give you a high payout. In particular, this gets progressively harder and the harder it gets, the lower the payouts. So while this person made $600'000 over 2 years, a repeat performance over the next 10 years, or so is exceptionally unlikely.

      An actually professional code security review does not depend on luck. It also does not try to maximize "bugs found". It looks at architecture, design, input validation, critical data-paths, etc. and more than half of the result will not be description of bugs found, but analysis of severity and conclusions to be drawn and even things that are not directly bugs, like critical data-paths that rely on one protection mechanism, configuration options that are hard to get right and breaks security (or remove one layer of protection) when done wrong, code structure that is misleading, security functionality in surprising places, etc.

      The whole focus on "finding bugs" is misplaced. You will never find them all and due to the randomization aspect of this, a different attacker may just find ones you did not find, regardless of how hard you looked. In fact, it is basically always cheaper and has a better result to re-implement with highly skilled and trusted people that thoroughly understand software security. Doing background-checks on these people and making sure they are satisfied with their jobs is actually much more important than to look at the code they produce. As so often in IT security, skewed and outright wrong ideas from bad movies are prevalent in the area of code security as well.

      --
      Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
  17. Two different and complementary things by raymorris · · Score: 1

    Code review and pentesting are two very different, yet complementary things. As you suggested, code review is likely to find a lot more, including things some people don't typically think of as "security" - points of fragility, for example. Code review is very useful, especially when done by people trained in security.

    Pen testing *after* code review is also very useful. It isn't unusual for code review to have a lot of detailed findings. As an analogy, looking at the internals (code review) might find that the deadbolt is is only held in with two half-inch screws, it's a cheap, crappy lock, and the gap between the door and the frame allows the lock to be shimmed. After you address those a pen tester looking at things from a different perspective walks right in through the side door. A concrete example is the OS. You might code review the application extremely well, then I shellshock right past it. "But the OS is out of scope!", you exclaim. So what. The bad guy and the pentester don't care about your review scope. We just walked right into your database.

    Two different, complementary things, both useful.