Do Data Center Audits Mean Anything?

1sockchuck writes "Data center service providers often tout certifications such as SAS 70, SSAE 16 and SOC 2 as evidence that they meet lofty operational standards. But some of these certifications are based on self-defined standards, and the entire situation is confusing and frustrating to customers, according to one critic, who says data center shoppers are poorly served by the jumble of acronyms and standards. Do these certifications matter when users are seeking data center space? Should they?"

Comments

Not really

[gweihir]

Now, if you get your hands at the detail reports, the audit result may actually tell you something, at least if the auditors are good. But the certifications pretty much only ensure minimal standards low enough to be meaningless.

Re:Not really

[Fluffeh]

Now, if you get your hands at the detail reports, the audit result may actually tell you something, at least if the auditors are good. But the certifications pretty much only ensure minimal standards low enough to be meaningless.

It depends on the purpose of the audit. If the purpose is to appease middle managers and the like, then the auditors (good or bad) will be able to read the request "We need to ensure we are certified for [insert current buzzword]." and see that this is nothing short of an easy way to make a costly fee. If on the other hand, the request is to find ways to break into the systems and comes from sysadmins or the like, then it is much more likely that the company wants to patch vulnerabilities.

Business is business. If a sales person sees easy money walking into the office, they will probably sell them overpriced and needless goods/services. If they see someone who knows exactly what it is they want, they will more likely give them exactly what they ask for and for a reasonable price.

Re:Not really

[eth1]

These aren't intrusion tests they're talking about but certification audits.

My experience with those (ISO, SAS, etc.) is that a company hires someone to write up a bunch of documents to match what the auditors want to see, and tell the employees where to find it. Then the auditors come and get told/shown what they want to hear/see so they'll go away and let us get back to real work. The documentation isn't looked at again by regular employees until the next audit.

Those certs are just like professional certs like MCSE, CCNA, etc. They don't really have any bearing on whether or not you're good at what you do, but they sound good to customers/employers.

It's better than no cert at all

[NemoinSpace]

not entirely unlike MSCE, but less so.

Re:It's better than no cert at all

[jhoegl]

Yeah, how dare he use actual logic instead of Microsoft Logic to acronym!

yes

[nimbius]

without data center audits thousands of datacenters across the country would have to forego tiny wooden plaques with things like "SAS70 CERTIFIED!" and "SSAE 16 READY!"
and I as a sysadmin would have to stop making the joke, "SAS70? oh thats for when we change the motor oil in the cloud."

Uses for Audits/Certifications

[ackthpt]

• - Waving in face of prospective customers - ' Yes w certainly a certificate of certification granting certitude!'
• - Finding things you actually did right
• - Finding things you need to fix or wallpaper over
• - Creating gainful employment for auditors, certifiers, pencil pushers, paper shufflers and rubber stampers.
• - Sell more seminars and books for a certification industry
• - Influence government to require certain certifications to keep an industry of auditing and certification on the gravy train for years
• - Give significantly less benefit to people who disagree with the need for dubious audits and/or certifications.

short answer: no

[Anonymous Coward]

I'm a work at a somewhat large financial services company that provides customer information to various other large financial institutions (chase, wells fargo, capital one, amex, discover, just to name a few). We receive this customer information from pretty much everywhere - those self same banks, government agencies, credit card companies, universities. Basically, if you've ever had a loan or grant, credit card, bank account, paid a utility bill, child support or been in prison then we have that data. Your address, phone number, social security number, bank account information, etc.
    The majority of this information is stored unencrypted on systems that are accessible to any employee, often with 777 permissions. While the majority of the systems are patched pretty regularly, many aren't. I recently had to convert over an old apache 1.3 server that hadn't been patched since 2006 - there's another similar server that is regularly used by outside contributors to drop off customer information.
    We have customer facing IPlanet servers that haven't been patched since 2004 - the software isn't even under support anymore.
    We have session recording software on our unix servers that is so ridiculously trivial to bypass that the company that sells it (centrify) should be ashamed to sell it.
Yet we've had PCI certification for 3 years, we've passed the SAS70 certification every time - they are rubber-stamps, nothing more.

Of course it matters

[ZouPrime]

Well, it certainly matter for regulation purpose. If you handle data that need to be covered under a specific standard (say, PCI), you'll seek out a certified data center. In this context, the certification isn't about security, it's about risk transfer. It's the provider who become liable if there's a breach if it can't show to have respected the standard properly.

Now as security references, they certainly have their problems. We can take solace in the thought that they help enforce the bare minimum at the very least. As a security professional, I would say their best benefit is how well they can be used as a big stick, "encouraging" management to perform necessary changes. It's a hard sell to convince an average manager to invest in security for the sake of security. But if there's a legal penalty associated with whatever standard must be put in place, as well as a big dollar sign attached to it, they'll suddenly start to listen. That's a language they understand.

These are NOT Certifications!

[DaCurryman]

There is a number of problems with how data centers make these statements and what people interpret. The main problem is that people say things like "SAS70 Certified". That is terribly bad wording. There is no such thing. The SAS70 (now SSAE16 or SOC1 report) is not a certification. There is no preset/predetermined criteria that is universal to all companies that receive such a report. Each report is specific to that particular company/data center. It's almost like saying I have a diploma as an independent study major. The next thing is that these reports are not intended for public use. These are auditor-to-auditor reports. They are meant for the auditor for a company that uses said data center (or other service provider) to rely on and not need to audit the data center itself. That is why auditors review these reports to make sure it contains the provisions it's looking for. Otherwise, they're going to go in and audit the data center. Companies that get such reports tend to use it as a marketing tool to show potential customers, when that isn't the purpose. To reduce some blame, I've known auditors guilty of telling data centers that they can do that so that they could convince the data center to pay for the service. Also, SAS70 was designed to reflect controls at a service provider that impact or relate to the processing of financial data, which would have an effect on the financial statements that the auditor is reviewing. Most data centers don't process data (the customers that host stuff there do and they need the SAS70). However, over the years, people have convinced themselves that because the data physically resides at the data center, they impact the financial statements and so they should get a SAS70. This is however, not really true, since with good security controls around the data, the physical hosting of it won't materially misstate the financials. It was for this reason that the AICPA split the old SAS70 into 3 separate services: SOC 1 (SSAE16) which is what the old SAS70 was meant to be, SOC 2, SOC 3. The latter 2 are geared more toward data centers and technology firms that don't impact financial data. The seals that are issued by the AICPA just state that you've had a report done. They do not speak to the content of the report. I could get a SOC report that just says "All employees are entitled to free breakfast". The auditor I hire will come in and test/verify that and then will sign-off saying that they agree. I now have such a report and can boast "SAS70 Certified" everywhere, which doesn't mean squat. It only matters to the company itself, the company that uses their services (depending on context), and the auditors of the company that uses their services.

If you want security and reliability...

[jafo]

Security and reliability are processes, they are not something you can do once and then forget about. So, yes, I would say that having regular audits are a useful thing. As far as whether these specific standards are useful, the facility we have most of our servers in we have been in since before their SAS 70 audit, and their procedures were good before, but there's a noticeable improvement after. Things like a man-trap with a live security person comparing you with your on-file photo before you enter the raised floor, 2-factor auth on all doors rather than just on the key doors, maintenance lock-outs displayed more prominently, EPOs installed (not a benefit to me, but they did put alarmed doors around the EPOs to prevent the common problems).

As far as it being "based on self-defined standards", I'm ok with that. I'm ok with the requirement being that they *HAVE* standards for certain things rather than dictating what exactly those standards are. One size does not fit all, but having standards for what you do, I have found in my own business, improves quality.

The answer you need to show your boss

[colonel]

Right here, pure gold: http://www.gartner.com/it/page.jsp?id=1400813

Read that 5 times, carefully, and then get your bosses to do the same. Seriously.

SAS70 is a *questionnaire* that the vendor completes, and then the auditors just go in and confirm that their answers are correct.

So I could say "we don't do backups" in my answer to the questionnaire, the auditors would verify that I didn't do backups, and I'd "complete" the SAS70 process (not a certification!) successfully.

It is the client that is resoponsible for reviewing the questionnaire and ensuring that the audited answers are sufficient for the needs of their business. That's called "vendor management" and is a core practice area in ITIL.