Recommendations for Third Party Security Audits?

palehorse asks: "I am a developer/DBA/etc for a very large State Govt. Agency on the East Coast. We have been subjected to an increasing number of break-ins and website defacements over the past few months. My boss has recently been tasked by our CIO to find a reputable third party (not us or our ISP) to come in and do a complete and independent security assessment/vulnerability analysis for us. Since I'm the guy who usually bugs folks about security, she tasked me to come up w/ a list of firms who could do this for us. and a plan on what to test for and how. I've done the whole Google search/ZD-Net search/etc, which has given me way to many folks who do this kind of stuff, from ISS and IBM on down. Consequently I wanted to get some feedback/suggestions from the Slashdot community on where to go from here."

"Please keep in mind that while we're a large government agency, we have a small and overworked IT staff who have no real experience in internet/web security, and who are just now getting into a serious web presence.

Here are the main questions that I have:

• Who have you used, and were they any good?
• What should we look for in evaluating who to contact and their proposals?
• What would you have done differently?
• What services should we ask for?
• How do we manage the contract to make sure we're not getting a snow-job?
• How do we use the result to get buy-in from management (who will probably need to lay out money for changes) and from the developers (who may be adamantly opposed to changing their systems, either for ego reasons or it's just a lot of work)?
• How often should we re-do these audits?

Again, I know there's a lot out there by the security firms themselves about this, but I'm really looking for a 'keyboarder-in-the-trenches' view as well. Horror stories are appreciated, since they may give us an idea what to watch out for."

Comments

Look at KPMG

[alen]

When I was a consultant for the US Army Corps of Engineers, they used KPMG. KPMG would do a monthly scan of the network and send us a report for changes we needed to make on servers and workstations. I think they also used them for the backbone network services, but not 100% sure.

Re:References

[jcoy42]

I've had some experience with the Root Group and was happy. They did a good job, and as the company I worked for was cheap, they are probably quite affordable.

The biggest problem was that the company I worked for didn't want to actually implement the suggestions because it was going to cost some money for things like a real firewall. :/

I've also had bad auditors come in, usually forced on the admin group by managment and sales staff. I would advise the following to avoid these types:

First, ask them ahead of time what thier requirements are to get started. If they say "root access", show them the door. There is no talent in a company that requires full access to see if you are vulnerable (Note: there is nothing *wrong* with giving them access as part of the audit, but they shouldn't be *starting* there).
Matter of fact, if they start with wanting to login to your servers, you can probably do better.

Make sure they understand trust trees.

Make sure they are familiar with your OSs and critical applications.

Ask for, and check up on, references.

It sounds like you are off to a good start. Having managment ask you to plan something will mean you can get a real audit.. I've been through several where the "audit" started with me handing out root access so they could run "crack" on the shadow files, followed by a find command to look for world writable files, etc..