Slashdot Mirror
How to Measure Security ROI?
UM_Maverick asks: "Does anybody out there have any experience measuring Return on Investment for security-related expenditures? For example, if management says that there's $1 million left in the budget, and you can either implement a new customer tracking system that is projected to save $300k per year, or implement a new security technology or process, how do you measure the return on the security spend, and convince them that it's at least worth considering? Googling for 'Measuring Security ROI' seems to just produce a list of articles that say 'Measuring security ROI is difficult.' Does anybody have some more direct experience or information?"
Why not grow it within your infrastructure?
If you buy a "1 million$" security infrastructure, you WILL miss something. Instead, build the security from the ground up, paired with each node.
If you have to "pay for it now", you're already too late.
I would start with figuring out what it would cost to fix broken systems, downtime, etc.
Then you can at least put a price on not being secure, and let management make a somewhat informed decision.
Measuring security ROI is proving a negative. Because stuff is not being broken into and information is not being stolen, the company is "saving" money by not losing money and gaining bad press.
Your benchmarks are what type of security issues you do encounter and how they are handled. For example, if a security package catches would-be intruders, that can be shown as a sort of ROI (as the package prevented X dollars of loss.) Another example is the cost of whole disk encryption. Having a laptop that is protected by WDE get lost, one could state that the encryption software (assuming its properly deployed, proper password and/or security token policies set, etc.) saved the company the loss of the data on the laptop.
Probably the best bet in proving ROI is how many, what type of, and the cost of, the breaches and incidents one had before a policy/software/infrastructure went into place versus afterwards.
The cost of a security breach is measured as the probability of an incident multiplied by the cost of the incident. Both numbers can be calculated surprisingly well, or at least made to sound plausible. Security software will reduce the probability of an incident. Calculate the difference. If it exceeds the cost implementing security, it's a good thing.
This is a basic formula used for all types of data security, including backup and disaster planning.
This is not my sandwich.
Spending money on "security" can mean a whole lot of different things. What type of security? What are you trying to prevent? I work at a company that produces certain security products, some of which have other applications as well. When you hand the CEO a nice graph of the DDoS attack that you got your ISP to filter for you when you subscribed to their service, show how many hours of downtime it prevented, and how much money went through the online store during that time, proving ROI is fairly easy. Other kinds of security are fuzzier. Stopping worms within your network saved IT X hours of rebuilding PCs and prevented those machines from being down this many hours times the average worker's hourly rate would have been unable to work during that time etc. and you can provide some estimates.
Before you get to that stage, however, you need to have specific security measures in mind designed to address specific security threats to your business. Some of these measures are easy to justify (need certification to do business with government agency Foo) and some are hard (better passwords make it harder for insiders to steal our customer database and sell it to Russian hackers who then use it causing a publicity problem and resulting lost customers).
It's the only way to be sure.
I guess I would give the PHB a potential cost of what breaches could happen and an analysis of your situation and what measures need to be done to prevent it.
i.e. If you are running a business that keeps SSNs, bank data or some other sensitive data you would factor in the cost of how many customers times how much it would cost if thier personal information were compromised. If you are in design/manufacturing, you could factor in R&D/loss of contract costs if designs were taken, etc. (not to mention press coverage and effects on future customers and the stock market for public companies.)
Also get any stories of breeches to a similar IT installation to show example that there is an issue.
It's not really an 'investment' as much as a reduction of liability, if the potential liability is less than the cost of the security it is a hard sell. But most likely it will be a fraction of the potential liability without it and even if you do get a breech after the security update it looks a whole lot better to clinets, the public and the press if you show a track record for keeping your security up to date.
"Enjoy what you're doing! If it becomes drudgery, you're doing it wrong!" - Jim Butterfield
In any case, if you worked for me and pulled a stunt like that I'd be starting to look for your replacement asap: I pay you to do your job, not to prevent other people from doing theirs.
-- the cake is a lie
You can't measure the probability of something getting broken into. There are a million ways to calculate it and all of them come down to making up a number in your head. Realistically, "vulnerability" (ie, probability of getting hacked) is a null value. Ignore it. Weight your data, whether it can be replaced, the cost to the business if it's compromised (unauth disclosure, corruption of the data, or denial of access). Then threat model how you could do any of those things to your most valuable data and where, your next most valuable data class, etc....mitigate from there. Also calculate reputation value. A really outstanding good ROI for security has nothing to do with numbers: It's called "I didnt end up on CNN or Slashdot today".
Security should be something that is considered from the beginning of design. Having said that, I know from experience that it isn't and that management tends to want to plug the hole after the boat sinks. That is, once something bad happens, you get all the money you want and all you have to say is "security". In order to get management to fund security efforts on their data networks, you have to have a good idea of what could happen to your network/data. The first step is to identify all the vulnerabilities to your systems. These include not only hackers and insiders, but also natural threats like earthquakes and hurricanes (these are mainly useful for disaster recovery solutions). Take those threats and multiply by the probability of that event happening. Probability of a hacker exploiting a known software vulnerability....pretty good. Hurricane in Kansas...probably not. Once you have these probabilities identified, then you have to measure the potential damage to the company. Will you lose all your data (destroyed, not stolen)? Will someone post/sell private data (company data or personal customer data) that was stolen. Were your servers totally destroyed and you have to buy new ones? Some of these have hard $$ costs to them. Others don't (think embarrassment and tarnished record). It's usually good to convey the "worst case" and the probability of that happening. If you make your case and still don't get the requisite funding...keep your vulnerability list and everything handy. Then if something does happen, you can point and say "told ya!" Atrivis
Informative? Informative would be explaining how he came up with accurate numbers for [Total Cost of Intrusion] and [Percentage Chance of Intrusion].
That's where the problem is in this whole issue. How much will it cost if we get owned, and how likely is it that we will get owned? If you can calculate those two data points accurately, then yes, it's easy as pie to figure out your ROI, but the problem is that figuring out the former, requires the services of a mind reader, and the latter requires the knowledge of all the weaknesses in your security and all the skills and motivations of those who want to break your security.
Sure, it's fine and dandy to pull some numbers out of your ass and plug them into an equation, but when you get taken and the cost is higher or lower than your predicted cost, then you had better hope no one holds you accountable.
ad logicam Claiming a proposition is false because it was presented as the conclusion of a fallacious argument.
"Risk analysis" is a formal approach to what you are talking about.
To a lesser extent "Decision Science" and "Influence Diagram" are also attempts at tackling this type of problem.
Google scholar will turn up many papers in this area and I know that my school (University of Virginia in the Systems and Information Engineering department) has some active research in "Cyber Security" and related security planning.
http://www.sys.virginia.edu/risk/
RudeDude
Perl/Linux/PHP hacker
You are never going to get money back from security investments, you are limiting losses.
That puts you into Risk Management analays, not Return on investment.
Think of it like going without insurance, worker injury prevention, or other loss prevention/mitigation.
But you get smelly fingers. You can't calculate the probability of a breach because you can't enumerate the threats or the vulnerabilities. How many unpublished zero-days are there for the stuff in your environment? How many hours of unplanned outages will you have this year? Consequently you are just pulling a number out of your ass. I agree you can get some good numbers for the cost of a breach. Not the probability. So you are evaluating a cost times a guess.
There is no security ROI. It is loss-avoidance. It is insurance.
There is a way to get a concept of the chance of a successful intrusion. There are actuaries that do create this data. Garner may be able provide a good benchmark, as can some industry associations. Heck, insurance companies probably are collecting good data to get a predictor.
I paid garner for a research paper to justify the purchase of one SAN solution over another. The second solution went TU a year later. I have met the guys who write the reports. They are pretty smart guys.
In God we trust, all others require data.
There are specific methodologies for modeling risks / threats and estimating their impact, that are used for justifying
Information Security budgeting.
Principles of Information Security is one book that I'm familiar with that has quite a bit of coverage of this topic. We used this for my course in Information Security a couple of years ago, and I found it pretty useful, FWIW.
Additionally, check this OWASP Page for some good stuff.
And finally, try googling for terms like Security Risk Analysis, Security Risk Assessment, and / or Security Threat Modeling.
// TODO: Insert Cool Sig