Slashdot Mirror
How to Measure Security ROI?
UM_Maverick asks: "Does anybody out there have any experience measuring Return on Investment for security-related expenditures? For example, if management says that there's $1 million left in the budget, and you can either implement a new customer tracking system that is projected to save $300k per year, or implement a new security technology or process, how do you measure the return on the security spend, and convince them that it's at least worth considering? Googling for 'Measuring Security ROI' seems to just produce a list of articles that say 'Measuring security ROI is difficult.' Does anybody have some more direct experience or information?"
Why not grow it within your infrastructure?
If you buy a "1 million$" security infrastructure, you WILL miss something. Instead, build the security from the ground up, paired with each node.
If you have to "pay for it now", you're already too late.
I would start with figuring out what it would cost to fix broken systems, downtime, etc.
Then you can at least put a price on not being secure, and let management make a somewhat informed decision.
Measuring security ROI is proving a negative. Because stuff is not being broken into and information is not being stolen, the company is "saving" money by not losing money and gaining bad press.
Your benchmarks are what type of security issues you do encounter and how they are handled. For example, if a security package catches would-be intruders, that can be shown as a sort of ROI (as the package prevented X dollars of loss.) Another example is the cost of whole disk encryption. Having a laptop that is protected by WDE get lost, one could state that the encryption software (assuming its properly deployed, proper password and/or security token policies set, etc.) saved the company the loss of the data on the laptop.
Probably the best bet in proving ROI is how many, what type of, and the cost of, the breaches and incidents one had before a policy/software/infrastructure went into place versus afterwards.
The cost of a security breach is measured as the probability of an incident multiplied by the cost of the incident. Both numbers can be calculated surprisingly well, or at least made to sound plausible. Security software will reduce the probability of an incident. Calculate the difference. If it exceeds the cost implementing security, it's a good thing.
This is a basic formula used for all types of data security, including backup and disaster planning.
This is not my sandwich.
At night come into the office and take out the server and steal any other info ....lock it up in some office where the boss wont look. When everyone arrives for work the next day and cant work due to the fact there is a missing server, and the police are being involved talking about taking all sorts of
equipment for forensics evidence, then pipe up and say that THIS WAS A DRILL...and let everyone go back on about their business. Once you are faced by the boss to explain your actions....just say that had this been the real thing....25 employees would have been without work and still gotten paid...
take their salary per day * 25 * how many days you think it would have taken to get everything back rolling again with a new server and new configs, and new passwords for everyone....this will be the total you should ask towards getting a better security system in place....including
cameras for the server room, a utility software (VMware???) made to replace images of machines
that were stolen from backups kept elsewhere, plus a utility to update all user accounts including
admins with new passwords and maybe even something to help secure the machines into place ( bolted down )
Guaranteed this will work to get your point across, possibly cost you a week suspension...but worth it in the end to show how far you are willing to go for your security!
count the time is takes to deal with forcing people to use passwords with a lot rules and making them change them a lot. As they will right them down / forget them a lot.
Until there is a major security breach, only a thorough security audit will give the organization an idea of how much a security problem can cost. If an audit demonstrates terrible flaws in security it should become obvious money needs to be spent on it. If the audit shows security is already reasonably tight then it's a tough argument to spend a lot more money on improving it.
Developers: We can use your help.
2. See how many times those people have been sued, payed victims off, or whatever it took to clean up the mess.
3. See what the cost was for #2.
(Divide #2 by #1) multiply by #3.
Get more customers is my vote, because I have never heard of folks who lose all of their customers data get sued and lose.
Don't me started on Universities! They're all FUCKTARDS when it comes identity security! Jesus Christ! Why TF do they have to use SSN, DOB, Name, AND address for student identity!!!!!!!!
(([Total Cost of Intrustion] * [Percentage Chance of Intrustion]) / [Costs of Security Measures]) - 1 = [ROI]
(($5,000,000 * .10) / $100,000) - 1 = 4
In God we trust, all others require data.
Spending money on "security" can mean a whole lot of different things. What type of security? What are you trying to prevent? I work at a company that produces certain security products, some of which have other applications as well. When you hand the CEO a nice graph of the DDoS attack that you got your ISP to filter for you when you subscribed to their service, show how many hours of downtime it prevented, and how much money went through the online store during that time, proving ROI is fairly easy. Other kinds of security are fuzzier. Stopping worms within your network saved IT X hours of rebuilding PCs and prevented those machines from being down this many hours times the average worker's hourly rate would have been unable to work during that time etc. and you can provide some estimates.
Before you get to that stage, however, you need to have specific security measures in mind designed to address specific security threats to your business. Some of these measures are easy to justify (need certification to do business with government agency Foo) and some are hard (better passwords make it harder for insiders to steal our customer database and sell it to Russian hackers who then use it causing a publicity problem and resulting lost customers).
It's the only way to be sure.
I guess I would give the PHB a potential cost of what breaches could happen and an analysis of your situation and what measures need to be done to prevent it.
i.e. If you are running a business that keeps SSNs, bank data or some other sensitive data you would factor in the cost of how many customers times how much it would cost if thier personal information were compromised. If you are in design/manufacturing, you could factor in R&D/loss of contract costs if designs were taken, etc. (not to mention press coverage and effects on future customers and the stock market for public companies.)
Also get any stories of breeches to a similar IT installation to show example that there is an issue.
It's not really an 'investment' as much as a reduction of liability, if the potential liability is less than the cost of the security it is a hard sell. But most likely it will be a fraction of the potential liability without it and even if you do get a breech after the security update it looks a whole lot better to clinets, the public and the press if you show a track record for keeping your security up to date.
"Enjoy what you're doing! If it becomes drudgery, you're doing it wrong!" - Jim Butterfield
Security isn't an add-on in this way, and it will (currently) always be bad advice to "invest extra $X in security". Security ROI only really becomes useful when you have decisions like: "We need X security, what is the best ROI solution".
Also consider that there is a large fuzzy middle ground between terrible "uwftpd, old sendmail" and very good "vsftpd, and-httpd" ... and even experienced user/developers tend to just group things into "terrible" and "everything else" (see, pretty much all web servers being clasified as "secure" even though a lot of them are poorly coded and Eg. Apahce-httpd and lighttpd have had remote security erratas). So the fact is, unless you fall into the "terrible security" group you are pretty much guaranteed to get a better ROI by doing anything else.
Note that when speaking about a company as against a specific product, this will just make everything even more fuzzy for 99.999999% ... so investing in security for ROI is going to be even worse.
ustr: Managed string API with ave. 44% overhead over strdup(), for 0-20B
You can't measure the probability of something getting broken into. There are a million ways to calculate it and all of them come down to making up a number in your head. Realistically, "vulnerability" (ie, probability of getting hacked) is a null value. Ignore it. Weight your data, whether it can be replaced, the cost to the business if it's compromised (unauth disclosure, corruption of the data, or denial of access). Then threat model how you could do any of those things to your most valuable data and where, your next most valuable data class, etc....mitigate from there. Also calculate reputation value. A really outstanding good ROI for security has nothing to do with numbers: It's called "I didnt end up on CNN or Slashdot today".
Security should be something that is considered from the beginning of design. Having said that, I know from experience that it isn't and that management tends to want to plug the hole after the boat sinks. That is, once something bad happens, you get all the money you want and all you have to say is "security". In order to get management to fund security efforts on their data networks, you have to have a good idea of what could happen to your network/data. The first step is to identify all the vulnerabilities to your systems. These include not only hackers and insiders, but also natural threats like earthquakes and hurricanes (these are mainly useful for disaster recovery solutions). Take those threats and multiply by the probability of that event happening. Probability of a hacker exploiting a known software vulnerability....pretty good. Hurricane in Kansas...probably not. Once you have these probabilities identified, then you have to measure the potential damage to the company. Will you lose all your data (destroyed, not stolen)? Will someone post/sell private data (company data or personal customer data) that was stolen. Were your servers totally destroyed and you have to buy new ones? Some of these have hard $$ costs to them. Others don't (think embarrassment and tarnished record). It's usually good to convey the "worst case" and the probability of that happening. If you make your case and still don't get the requisite funding...keep your vulnerability list and everything handy. Then if something does happen, you can point and say "told ya!" Atrivis
Being in INFOSEC, and coming from both sides (security vendor, large enterprise) their is no easy solution. A malicous attack can be a loss of information, which can be shown by the value that information is to the company. If its higher than the cost of implementing a protective measure, then you can see the difference easily. The hard one is if the malicous attack takes down your network or e-commerce sight or email. DoS attacks have far reaching effects and cost burdens depending on the attack. What is the cost of not getting email for the guy on the help desk as compared to the cost of the VP or CE* getting an email? What is the cost of lost time in man hours for each indivual on the network? All these factors will be subjective and depend on what your company holds ultimate value for.
AFAIK, you consider security the same way you consider insurance (or as an insurance complement): How much your business continuity's worth? Should you be hacked/DDoSed, how will it affect your revenue? How much money would you need to get it all back online? You'll also need to consider how intrusive your security is: data backup solutions sometimes require you to stop services while they are processed. If you want backups to be non-intrusive, again, you'll consider how much these stopped services cost. Same goes for services availability (clustering, replication, off-site disaster recovery plans).
In short : your security investments are business specific. No easy rule of thumb in here, I'm afraid.
Hope this helps.
Economists have long had a method of measuring "expected value" which is the sum over all outcomes of the probability of that outcome times it's value.
So in this case the value of the security software is:
(1 - Pb) * 0 + Pb * VA
Where:
Pb = probability that it saves you from getting broken into
0 = value if you don't get broken into
VA = value of your ass
Intron: the portion of DNA which expresses nothing useful.
I would rank and identify the projects you feel are most needed in the security area and then do some research and bring the ones you can make a realistic case for. Management likes numbers but keep it concise and honest. They also like to think they, being excessively smart, hired excessively smart people so cover all your bases beforehand.
If you can say "We could buy this system which severely decreases the chances of X happening. When X happened to Bob, Inc. they lost eleventy-billion dollars in revenue, downtime, lost productivity, etc... This system has positive feedback from these five major corporations who have successfully implemented it accross their enterprise in an average of 4 months and since implementation, none of them have had X happen to them. The realistic chance that we're going to be targeted by something like this within the next five years is about 10%."
Solve for X... and Bob, Inc, "eleventy-billion", 4 months and 10%.
Also remember that while you're doing a lot of estimating on the risks/savings, so are the people who are arguing for the new ERP system or what have you. You may not win the argument, but when X happens at least you have "I told you so".
Kneel before Sig!
..of buying health insurance?
(Yeah, health insurance has probabilities you can use to calculate, more so than security, but that's the mindset you should have. Maybe a better analogy is, what's the ROI of eating healthy and exercising?)
TJX (TJ Maxx, Marshalls) will have recent data.
You are being MICROattacked, from various angles, in a SOFT manner.
"Risk analysis" is a formal approach to what you are talking about.
To a lesser extent "Decision Science" and "Influence Diagram" are also attempts at tackling this type of problem.
Google scholar will turn up many papers in this area and I know that my school (University of Virginia in the Systems and Information Engineering department) has some active research in "Cyber Security" and related security planning.
http://www.sys.virginia.edu/risk/
RudeDude
Perl/Linux/PHP hacker
Proper configuration, proper coding, logging, and timely patches cost hardly anything.
Antivirus software attempts to substitute for user education, and sometimes slows down systems, reducing productivity. But some users never learn.
IDS software warns you about threats that should have been blocked by proper configuration. Except that it's nice to find out when an employee brings their virus infected laptop in and connects to your network, maps network shares, etc. I always figured Snort was the best IDS out there, and free, but my experience is limited.
The expected return is simply the sum of (cost of threat)*probability for each threat blocked, not that estimating either cost or probability are easy. Both tend to increase with company size.
In dollar value, insider threats can be much larger than any virus, and often get the least attention. Some client-server solutions, where a client-side program connects directly to a database server, often with a hardcoded or easily retrievable password, tend to be easy to exploit. Every commercial ERP and POS system I've ever demoed or administered was built in this fashion. Getting a list of customers' credit card details is usually straightforward. I saw one written in COBOL that had a super secret hardcoded admin login which you could probably guess in 20 tries with no clues. Even if you can't find the login, you can probably query what you want with a custom report.
Cost of a case vs. probability of it happening.
... well, consider the machine down for at least a day, and the data it contains lost.
Unfortunately (or thankfully), a lot of companies don't have experience with a case actually happening, so they can't easily figure out the probability. The threat of viruses/trojans is actually more important for many companies these days than an actual targeted attack (unless they guard some important business secrets in their servers). The chance of this happening can be gotten fairly easily from a security company, they usually have the numbers. The cost per case is
In other words, it would maybe be a good idea to knock on some doors of security companies, ask them for numbers and wave the paycheck in front of them, should they be convincing.
We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
You are never going to get money back from security investments, you are limiting losses.
That puts you into Risk Management analays, not Return on investment.
Think of it like going without insurance, worker injury prevention, or other loss prevention/mitigation.
What he said. There are several reasons the situation is this bad.
Insurance companies can tell you how likely a fire is and how much it costs to clean up and rebuild after one. They have the numbers to justify "loss prevention programs" and to justify giving you discounts for alarm systems. Finance people know all about this.
For security incidents those numbers simply aren't available. It's hard to cover up a fire, but lots of places hush up security events. The costs are partly intangible (how do you put a number on winding up in the newspaper for leaking 50,000 Social Security numbers?). Unless you're a huge company collecting data internally and using separate charge numbers for incident responese, you have to guess at the numbers that would go into ROI. Finance people can spot guesses from a mile off. Talk security ROI to them and they'll know not to believe you.
The approach some people are taking, at least in my part of the world, is to build a believable estimate of worst-case costs (fumigate and rebuild servers, pay for credit reports for all the customers) and then make the budget case to the level of management that would have to authorize funding the cleanup expenses.
For a new deployment, you have to take into account what is in place, what are the weaknesses, and how they are being
r ucture/33200565-b133-4eed-8c05-c6f35f8f60b6.html
addressed by the new thingum. google around for household names with breaches like so:
http://www.itworldcanada.com/a/Enterprise-Infrast
That article talks about basic things like establishing a perimeter. IF your company does not have a decent DMZ defined,and proper
safeguards wrt Intrusion detection, and properly walling off remote services. If people are sending credit card information via telnet, then you probably want to work on security. A gadget rarely solves anything. security is 99% about people and processes because you have to cover all the bases. The bad guys just have to find one weakness.
But you get smelly fingers. You can't calculate the probability of a breach because you can't enumerate the threats or the vulnerabilities. How many unpublished zero-days are there for the stuff in your environment? How many hours of unplanned outages will you have this year? Consequently you are just pulling a number out of your ass. I agree you can get some good numbers for the cost of a breach. Not the probability. So you are evaluating a cost times a guess.
There is no security ROI. It is loss-avoidance. It is insurance.
Hire an actuary.
(Though I have no idea where you'd find an actuary who would be able to answer your question.)
http://outcampaign.org/
There are specific methodologies for modeling risks / threats and estimating their impact, that are used for justifying
Information Security budgeting.
Principles of Information Security is one book that I'm familiar with that has quite a bit of coverage of this topic. We used this for my course in Information Security a couple of years ago, and I found it pretty useful, FWIW.
Additionally, check this OWASP Page for some good stuff.
And finally, try googling for terms like Security Risk Analysis, Security Risk Assessment, and / or Security Threat Modeling.
// TODO: Insert Cool Sig
I would start with figuring out what it would cost to fix broken systems, downtime, etc.
Right on!
This is not a situation that can be analyzed in terms of ROI; ROI is the wrong tool for this work. Writer of TFA should check out "Risk Management" for a start. That is what you want to be doing: providing the corporate officers with a report that says "Here are the risks measured in dollars of potential loss; here are the odds we face on each of the risks; here are some strategies we could use to mitigate these risks; and here are the costs of adopting each of the strategies".
If I was thinking of taking on this kind of project, I would go first to Accounting and see if I could enlist their aid. A lot of this is in their baliwick. Fortunately, it seems most bean counters enjoy the game of this kind of cost accounting. On a different matter, I once sat down with a financial officer and my data flow diagram, and by tracing through it together, we were able to work out how many of what kind of personnel were involved in each of the processes. It wouldn't have been hard to estimate the payroll costs for each hour of process activity, and usually the payroll costs are the biggest costs.
ROI. Wonderful buzzword, usually meaning "I don't have a clue what I'm talking about but I want to sound impressive".
Put simply you can't earn a return on a cost, only on an asset. Investment doesn't mean "put money into it", it means capital expenditure to acquire an asset. You invest on the basis of an expected return at a given level of risk associated with the asset. ROI is one of the measures that can be used to assess the attractiveness of the investment.
These are definitions. You can't substitute definitions of "return" and "investment" from other domains into the phrase "return on investment" and expect the calculation to produce a meaningful answer.
Another definition: risk is deviation from the expected outcome. In order to assess risk you need to know the potential risk events, their frequency and their magnitude. Then you can manage the risk by protecting against events, insuring, or doing nothing and hoping your business can take the losses.
Go and read some books on risk management and learn the appropriate measures of risk. You can probably start with the Wikipedia. Don't try and fudge it by using investment management measures, which are inappropriate.
You can't make money by improving your security. You can reduce costs (associated with your existing security system) and you can reduce the frequency and/or magnitude of risk events. Of course if you haven't identified the risks in the first place you don't know if your security is going to be effective, or how much those risk events would cost, or if insurance is a better option than prevention.
i-name =twylite [http://public.xdi.org/=twylite], see idcommons.net
One thing you might also want to consider is that there are certain side benefits that can also be calculated. If, for example, you put in a system that allows remote administration, so you can push out packages to machines en masse, instead of having to visit each and every machine individually, you've got a tremendous reduction in man-hours. Management types love that, and it's far easier to calculate than the chances of being attacked.
ROI is essentially a ratio measuring a payback period, which can lead to distortions. Say you have two projects. The first has an investment of $1,000,000 and saves you $100,000 per month. The second has an investment of $100,000 and saves $10,000 a month. Both have a payback period of 10 months (100,000/10,000) and both have an ROI of 100% (100,000/10,000). Which project do you do? Assuming that you can afford to both project, which do you do? Based on this information, you would do both. The missing element is cost-of-capital, which where security comes in.
Look at it this way: You are looking to roll out an SSL-based VPN that will reduce your ongoing remote access costs by $200,000 per year for 2,000 users. Youre concerned however that one of the main drivers for the project is that users want to login from un-trusted web kiosks. You think doing using SSL instead of a client-based solution is more risky than going with IPSec, but how can you analyze it? If your companys WACC is 10%, then the value of $200,000 should be $2,000,000. But this project is far riskier than youre companys main line of business, so the project should capped at a much higher rate. If you use 20%, then the value is $1,000,000.