Security Analysis Reports for Managers?

chaffed asks: "I've been tasked with translating a security analysis report for our powers that be and ultimately for our auditors. The manager's are not technically savvy, but they aren't PHBs, either. To what depth should one descend into Information Security and Technology topics? More generally, how would a technical person relate to a non-technical person? Should all technical terms be defined or just cryptic ones? What assumptions are reasonable to make about the reader (Non-Technical Managers)? What physical format should an analysis take, bulleted points or in depth discussion?"

Comments

Resources

[luder]

More generally, how would a technical person relate to a non-technical person?

I know a few guides on that one, seems to work pretty well.

To answer your question

[Anonymous Coward]

• Managers

• Like

• Bullet Points

Re:To answer your question

[Anonymous Coward]

The Problem is that
* Managers
* Like
* Your ass at stake

The only reasonable approach would be:
"I'm the security officer. You can be confident about me, so I'll give you no technical explanations *at all*; just what has to be done, or you can distrust me; then you should fire me and hire a new one of your confiedence".

Really, it is the only good approach.

Now: you dont believe it and:
* Go into a phb-ized version of your technical analisis. One one hand, they will disrespect you. Your job can't be so important if even non-trained people like them can understand it; your advantage is that you had the time to read some minor low-level details about your work; apart of this, they could be doing it themselves. But then something goes wrong. Then you tell your managers you told the risks. They cover by telling you didn't go into enough technicalities, so they wouldn't grant the OK under sounded ground (and rightly so: you really can't simplify your bussiness; you just can cover some facts that sooner or later will return to reclaim you their proper place). Anyway, it's your fault, not theirs (it's not an imagination. Don't you remember the "powerpoint-slides-and-the-shuttle-goes-boom" case?)
* Go into the deep technicalities and in four minutes your managers will tell that's not what they were looking for, it's too boring and anunderstandable so you are at fault *and* nothing goes approved ...but since its your fault, not theirs, anything awry that may happen is still your head, not theirs.

There was the case about the pharaoh asking Euclide for a shortway into learning maths. There's no shortways, not even for kings, Euclide answered.

Well, I'd bet that should be your answer too.

here is what I do....

[cavtroop]

I sort the report by severity, and calculate statistics from that. the first few pages are the 10,000' view - i.e.: we have 7 systems with level 5 vulnerabilities, 38 systems with level 4, etc. etc.... Then, on the following pages, I break down the report into the nuts and bolts - that lets the managers that want just the overview to stop reading after the first few pages, and provides detail for the managers that want it. is that what you are looking for? pretty basic, actually...

Re:here is what I do....

[SatanicPuppy]

Absolutely. Put in an executive summary, and you can pretty much fill the rest of the report with hardcore tech jargon. Generally I put in a summary page, then a mid-level summary that is light on the jargon, and then I just append a slimmed-down version of the raw data to the end.

It generally meets with approval.

From experience

[Opportunist]

Make it as graphic as possible. I mean it literally. If there's a way to do it in a chart, do it. Managers also tend to read summaries first, and more often than not, only. So make sure the summary gets the point across.

They don't care about technical issues. They care about cost, threat level and benefit. Make sure you cover that bases. Don't try to construct too many "if" sentences, since they'll be brushed away with "won't happen, don't care" too easily, even if that "if" does actually mean "when".

Explaining terms is fruitless, imo. They will skip over the parts they don't understand rather than look up terms in a glossary. The same is true for a lengthy prolog trying to explain terms. And don't try to create hyperlinked (or otherwise electronic) documents to make looking up things easier. They'll just print it and your time is wasted.

As said before, they don't care about technical details. They don't care where a potential attacker comes into the system. They care about the questions:

How do we prevent it?
How much time (in manhours/mandays) does it take?
How much does it cost?

Make sure you cover those 3 questions. Preferably in the summary.

Re:From experience

[techno-vampire]

Don't try to construct too many "if" sentences, since they'll be brushed away with "won't happen, don't care" too easily, even if that "if" does actually mean "when".

In that case, if you mean "when that happens..." write it that way. Don't say "if" because they'll think it will never happen. "When" tells them it will happen, sooner or later. Say what you mean, not almost what you mean.

Re:From experience

[bpb213]

I want to highlight a couple of your points:

"They don't care about technical issues. They care about cost, threat level and benefit.

Exactly.

However, I want to change around what you should present. Present the following:

1) How much does it cost us if we do nothing
2) What do we need to do to eliminate or reduce the cost of 1.
3) How much will step 2 cost. List both costs for eliminating step 1, as well as costs for reducing the majority of 1.

And to stress: SUMMARIZE, SUMMARIZE, SUMMARIZE. Prepare to back up any claim you make, but keep the summary short, sweet, and too the point.

Also, be prepared to compromise. You never seem to get 100% of a solution from a manager. Which is why you need to present the 75% or 90% solution, in addition to the 100% solution.

Re:From experience

[toadlife]

"Also, be prepared to compromise. You never seem to get 100% of a solution from a manager. Which is why you need to present the 75% or 90% solution, in addition to the 100% solution."

In the public sector it's similar, except you take the 100% solution, double it, and then present it as the 100% solution.

In general;

[GoofyBoy]

Everyone is different so I would take a manager and ask for feedback on a draft.

But in general;

>To what depth should one descend into Information Security and Technology topics?

Enough to make it clear as to why the topic is important and what impact it makes on the company. And not too long to make people bored.

> More generally, how would a technical person relate to a non-technical person?

With clear accurate words and descriptive and varied sentences. Interesting examples are good too. One of the best documents about a technical/complex topic I've read is at http://www.torontoinquiry.ca/ It was a long and boring inquiry about computer leasing, goverment procedures and its long and detailed. But reads like a trashy novel and very accessable and understandable. I enjoyed reading it and afterwards I felt I knew what had happened.

>Should all technical terms be defined or just cryptic ones?

Every single one of them in the glossery.

>What assumptions are reasonable to make about the reader (Non-Technical Managers)?

That they have at least a high-school level reading level and they need to know the information contained in your document.

>What physical format should an analysis take, bulleted points or in depth discussion?"

Yes. Use what ever you think you should need to use to clearly covey information. Formatting and layout are just tools.

I think Dilbert explained it best

[hayden]

"Well, something that you could never comprehend conflicts with something that you'd never understand."

It's Security

[DivineOmega]

It's security, thus you must make the most important points (i.e. those of greatest risk) the most prominent in any report and make them easy to understand. It'd recommend first bullet-pointing each security aspect in categories such as severe, medium and minor issues.

After this categorisation, it would be wise to describe each point in more detail in an after section using non-technical language, but making it obvious what the implications of ignoring these security issues could be. This should again, be priotised on the most major of these points.

If a manager is unable to understand a point or worse misunderstands, he may consider it trivial. And obviously, this could be disasterous over the long-term.

World's vaguest Ask Slashdot. Evar.

[Gothmolly]

So I have, like, some security data. And some managers, and like, it was really good data, and I'm afraid if I give it to my managers, they'll go like, beep beep beep, and ask further questions, which I won't be able to articulate answers to, and then they'll hire $GARTNER_PICKED_CONSULTANCY_DU_JOUR and I'll be out of a job. Bummer.

General plan here

[swordgeek]

Here's how to write it as if you were an auditor. When it gets to the auditors, they'll eat it up.

First of all, the executive summary. "We are mostly secure/insecure, with (n) critical action items. Of these, (x) can be implemented with little effort or cost, (y) will require substantial effort and/or cost, and (z) will require a fundamental change in the way we do business." Actually, this breakdown might be a bit detailed for an ES. Yes, really.

Then provide the background: "The internet is a scary place. (n)% of security breaches come from inside. Personal laptops can sniff unencrypted traffic. Passwords are easy to hack. Security breaches can undermine us in some specific way, or cost $xxxMM. etc.."

Now the specifics: "Preliminary analysis of our network has uncovered some critical/significant/minor security flaws. These are blah, blah, and blah, in increasing/decreasing order of severity/cost-to-fix. A detailed analysis of these flaws is as follows:
(flaw1)
(flaw2)
(flaw3)
(...)
The analyses should be broken down in a fair amount of detail, with technical terms defined in a glossary at the end of the report. Each one should contain the cost-to-fix and the cost-of-breach if possible, as well as the likelihood of a breach. Having a DMZ mail server taken down by hackers might be a huge pain in the ass, but ask (i)will it actually cost the company that much money in lost productivity, (ii)how likely is it to happen, and (iii)how much will it cost to improve? Alternatively, a disgruntled admin can potentially destroy your data centre--downtime at (d) dollars/hour, plus the cost of lost data since the last tapes. A third alternative is loss of proprietary data to a competitor, which might be bad, or might be enough to shut the company down permanently. Be VERY CAREFUL here, though: If you're writing a security analysis, then don't stray into trying to build an entire DR plan. Seriously. Don't.

Summary: Exactly that--summarise the detailed analysis, ordered by the the cost/benefit ratio. Make sure that the difficulty of implementation or added risks are considered as well. Remember that at this point, you're just summarising the data, not yet doing the...

Recommendations: "Based on the above data, we recommend implementing blah and foo immediately. These provide some/significant improvements in security, can be achieved with a minimum of effort or cost outlay, and carry little/no risk of introducing new problems. In the 1-3month timeframe, 3-6 months, 6-12..." That sort of thing.

Then of course, the glossary.

Don't ever forget: Security weaknesses are the cost of doing business. For example: Moving from telnet to ssh provides a significant benefit, and allows you to keep working. Shutting off all interactive logins doesn't provide much further benefit, and most likely substantially interferes with the company's ability to do work. Limiting ssh access to a few client boxes may provide a security benefit (hard to quantify), but may also increase the administrative overhead enough to make it not worthwhile. All managers and techs much understand that security isn't an absolute goal--it's a degree of risk acceptance. Eliminate all unnecessary risks (security or otherwise) be aware of all the necessary ones, and mitigate the risks as much as possible.

Risk versus CounterMesure cost

[Anonymous Coward]

Your manager need to know what the vulnerabilities, what is the risk of exploitation, how it will cost if the exploitation succeed and how cost the counter measure to mitigate this risk. Think a your public webserver that can be defaced because you forget to put a security patch, the cost of the security patch is maybe few hour time, but if it's exploited the time to shutdown, the time to recover, loss of customer confidence, etc.. will be very expensive. But sometime you have specific software with security problem and the cost will be very high (because you didn't include security fix in your maintenance contract) and the risk will be lower (no public access). All is about money, cost versus safety, no report to the manager should be technical, but only to explain in a high level what are the vulnerability (both software, hardware, physical and human), what is the risk, and how you can fix the situation. try to put critical first, sometime your critical is not the manager critical (easy flaw in the billing versus possibility of a remote overflow on a custom CGI) and try to allocate budget to the fixe, and try to provide alternative if it's too expensive (don't elimate the 100% of the risk but lower it to a acceptable level). Dont forget, no manager can do the difference between some apache overflow, php forum exploit or SQL injection but the will understand "unauthorized access to our server with possibilities to execute untrusted command" and "altering the integrity of our database".

Re:Risk versus CounterMesure cost

[maciunas]

Apart from poor formatting and errors in spelling etc, I have to agree with this.

Having done many such reports (as an independant "audit" process), I just have to add one thing that goes against the general flow of the postings so far. Don't dumb-down the report - people who have management roles are generally literate and have active brain cells. They need to make their own call on how important things are, they are looking for your narrow technical viewpoint and will add that to the other narrow viewpoints from the rest of the organisation.

So you need to make the important issues clear and upfront and with some kind of cost of not-doing. I've never written a report with $ costs, just in expected down-time to fix etc. I've always found this quite effective in getting the point across - there is the list of critical must-do's and then the should-do's with rough time-and-materials costings.

I think most of my "should-do's" were done and the must-do's scared the pants off some of the managers - on the basis that the costs for the lower priority items were so high.

Probability, avoidance, resolution

[Bogtha]

Security is a complex business, but the effect each potential problem has on an organisation can be summed up in simple terms.

Probability: What is the chance of a particular problem actually happening?

Avoidance: What would it take to avoid/reduce the chance of this problem occurring? How much will this cost? What will the effects be on productivity? What about morale?

Resolution: If the problem does happen, what will it take to fix it? How much will that cost? Downtime? Legal liability? Bad press?

Those are the three main variables a manager needs in order to decide whether to take action on a potential security problem. For example, if something costs a lot to mitigate, but is very unlikely to happen, then it probably won't be worth doing anything about unless the costs for fixing the problem are extreme.

You only need to go into the technical stuff in order to explain the above. You don't have to explain how the attack works beyond its immediate ramifications for avoidance etc. In-depth discussion will be skimmed, so break stuff into bulleted lists, charts and tables wherever it makes sense to do so, with a clearly marked summary for each section, so even the laziest PHBs will have an overview.

Oh, and not specifically work related, but if you are going to be writing stuff for other people to read in a professional capacity, then making obvious grammatical errors is unprofessional.

Re:Probability, avoidance, resolution

[MrNougat]

(Mod this redundant if you must.)

That's basically the formula laid out in Fight Club.

A new car built by my company leaves somewhere traveling at 60 mph. The rear differential locks up. The car crashes and burns with everyone trapped inside. Now, should we initiate a recall? Take the number of vehicles in the field, A, multiply by the probable rate of failure, B, multiply by the average out-of-court settlement, C. A times B times C equals X. If X is less than the cost of a recall, we don't do one.

If it costs less to recover from a failure than to prevent the failure in the first place (taking into account the number of predicted failures over the expected lifespan of the prevention implementation), you plan to recover from the failure instead of preventing it.

Think beyond a 'security gap'

[MrNougat]

Make sure the powers that be understand the concept of driving an exploit into a single small security gap, and using that to bootstrap into higher and wider levels of access. Don't let them think that because they've assigned budgeting to address the top two tiers of security holes that they can let the bottom three tiers slide. Surely, the most glaring gaps deserve the most immediate attention, but that doesn't mean that you can rest on your laurels after you've tackled the big issues.

I see too many non-technical business leaders want to implement some technology or other, then expect it to go on and on forever without any further attention. I have to think that there are companies out there that have the same view of security. "Well, we did that already, so we don't have to do it again."

Maybe all that doesn't apply to you, what with the non-pointiness of your bosses' heads.

OR

[WedgeTalon]

Or you could just ask your managers this question. I imagine that they'd be able to give a more accurate answer than a couple thousand random people.

But maybe that's just me.

Re:OR

[karlto]

Or if you are really feeling adventurous, you could use a standard formal report structure similar to those that have already solved the problem of reporting technical information... Synopsis => Findings => Conclusions => Recommendations...

To put it another way

[Beryllium Sphere(tm)]

>Make sure you cover those 3 questions. Preferably in the summary.

Deliver a risk analysis rather than a security analysis. It's what most people really want and it's what HIPAA clients explicitly need according to statute.

Taking that example

[Beryllium Sphere(tm)]

>Limiting ssh access to a few client boxes may provide a security benefit (hard to quantify), but may also increase the administrative overhead enough to make it not worthwhile. All managers and techs much understand that security isn't an absolute goal--it's a degree of risk acceptance

This is where technical depth comes in. The next cheap big improvement to suggest to someone running ssh is to disable password authentication (especially given all the brute-force login attempts we've seen). Then everyone who needs ssh access gets a nerdstick to put on their keychain (if they lose it they lose their house keys) with the private key.

>All managers and techs much understand that security isn't an absolute goal--it's a degree of risk acceptance

A lot of managers are already there -- they live in a world of tradeoffs and many of them know it.

Re:Taking that example

[swordgeek]

"The next cheap big improvement to suggest to someone running ssh is to disable password authentication (especially given all the brute-force login attempts we've seen). Then everyone who needs ssh access gets a nerdstick to put on their keychain (if they lose it they lose their house keys) with the private key."

This is definitely a win from a security point of view, but the cost analysis is a tough one. How much does it cost for the keyfobs, the software to manage them, and the time involved in tying authentication to them? How much more secure does it make your system, and is it quantifiable? Put another way, let's say the likelihood of a break-in without the things is x%/year, and the cost of a break-in is likely to be $y dollars. If you look at a long view of five years and find that the things cost the company more than y(1-((1-x)^5)), then it's generally not worth it. Furthermore, if it's a small cost benefit, then convenience and privacy should weigh-in. (One of my contracts just implemented fingerprint scanners on all of the PCs across the company--stupidest thing I've ever seen, and I've so far refused).

"A lot of managers are already there -- they live in a world of tradeoffs and many of them know it."

No doubt about that, but both managers and techies can fall into the binary solution mode: Something is either broken, or it's fixed. It's either insecure or it's secure. If we need to be secure, how much will it cost? Don't give me shades of grey, give me a PO that I can sign!

No slight intended against managers by my comment--I just meant that it is THE MOST IMPORTANT part of security, and something that doesn't often get applied to computer security as often as it should. It's one of those 'obvious' points that needs to be put in papers for the same reason that all of those 'obvious' technical definitions need to be there. Someone might not realise it, or they might have forgotten what the real target is.

Ever noticed how much security is like dentistry?

[Beryllium Sphere(tm)]

Small problems can open the door to large problems. You have to pay daily attention to the small problems to control them. Periodically you need a professional to look things over. Problems can be agonizing and/or make you look bad.

And none of it's any fun to the patient.

AC Suggestion

[Anonymous Coward]

Use bullet-point list, but elaborate on each. Keep it succint and focused on managerial point of view, however - you want to get points across, not show off your depth of knowledge. Analogies help, but make it clear that extending them is dangerous. Little bit of scare tactics (with stats, news clipping, etc.) can be helpful on points you feel strongly, but this is risky and should be used sparingly if at all.

And on points you do not feel completely certain, let them know that you don't know. This often times actually boost the level of confidence they'll place on you.

Actually, the above applies to most situations where you are making a presentation, not just tech-to-management.

Nice timing...

[wildblue7272]

I actually just turned in a security analysis to my boss yesterday. (We are a small non-profit business.) After giving it much thought, I decided to go into a little more depth than what I knew he would understand, but broke it down (mainly bullet format) as much as possibile. My goal was to draw questions out of him, that way we could have a meaningful discussion about INFOSEC. (Plus it never hurts to flex the ole' brain muscle around the boss every now and then.)

You can be technical... but don't forget what the boss TRUELY cares about: the big picture. Let him/her know the current status of everything... what really needs to be done to preserve (or establish) INFOSEC... and how much it's gonna cost.

Good luck!

I happen to write these reports every so often...

[Cybersonic]

Ill make this short, informative, and somewhat dumbed down, just like the type of report they are looking actually for.

Go here and read: sans.org/rr

They want a few powerpoint slides worth of information in a doc/pdf really... Lots of pictures and graphs. Highlight the risks and list the tasks needed to mitigate them.

Try to cover your own analysis of the products you have in place to protect your company.

• Network-based Firewalls
• Network-based Anti-Virus
• Network-based IPS/IDS
• Network-based Anti-SPAM
• Host-based Firewalls
• Host-based Anti-Virus
• Host-based IPS/IDS
• Host-based Anti-SPAM
• Patch Management
• Vulnerability and Application Assesment
• VPN (IPSEC and/or SSL-based)
• Authentication (LDAP, Radius, 2-Factor, etc...)
• Anti-SPAM
• Event Management
• Logging Servers
• Content Filtering
• Wireless Security

I hope you have at least some idea of a plan for each of these areas...

Other method- PRSC

[Marxist Hacker 42]

The other method is a rather simple table:
Problem, Risk, Solution, Cost

This is what managers actually care about. A one sentence description of the problem, the risk of missing or stolen data as a percentage risk, a one sentence description of the solution, and the cost of that solution. That gives them what they need to know to work up your budget- and you implement what they want.

Also from experience

[diverman]

> How do we prevent it?
> How much time (in manhours/mandays) does it take?
> How much does it cost?

I think that some other things are more important to a manager:
What is the risk to the business? (regulatory violation, monetary loss, etc.)
How much will a breach cost? (fines, downtime, system replacement, recovery, lost business, loss of competitive advantage)
What is the risk of occurance? (how likely is a breach to occur?)

In all honesty (you may disagree), I think these questions come way before those you posted. Before a manager is going to ask about how to take action, they're going to want justification that they need to bother taking action. I agree that the answers to these questions (at least in short form) should be in the executive summary of the report.

-Alex

you should...

[ohzero]

speak klingon. klingon is not technical.