Should Auditors Be Liable For Certifications?

dasButcher writes "Enterprises and mid-size business rely on auditors and service providers to certify their systems as compliant with such security regs and standards as PCI-DSS or SOX. But, as Larry Walsh speculates, a lawsuit filed by a bank against an auditor/managed service provider could change that. The bank wants to hold the auditor liable for a breach at its credit card processor because the auditor certified the processor as PCI compliant. If the bank wins, it could change the standards and liabilities of auditors and service providers in the delivery of security services."

Comments

Oh, this sounds like a good idea...

[fractoid]

TFA makes a very good point:

What will be interesting about this lawsuit is how the court assigns responsibility for a breach at a certified business. Audits, by their very nature, are point-in-time or snapshot checks. They cannot account for the dynamic variables of business and IT operations that may weaken security over the long-haul.

If they win this lawsuit, they're setting a dangerous precedent - anyone who at any stage has certified a system as secure becomes responsible for its ongoing security, and can potentially be held liable for stupid user errors by users of that system.

Re:Oh, this sounds like a good idea...

[Renraku]

Inspectors of things like elevators are not responsible if their target checked out at the time of inspection, and later failed. For example, you could sign off on the construction of a bridge or an installation of an elevator because everything looked good, but when the bridge company doesn't maintain the bridge properly or the elevator company fails to do the same, the inspector is not held liable, even though they were certified as good.

Auditing a network should be the same way. Of course, an auditor should NOT be held responsible for undiscovered bugs or holes in software. Instead, their job should focus on general security. It would be like a bridge inspector trying to certify a bridge when the gravitational constant of the universe were in a state of flux. How do you guarantee that steel is the best material or that the iron won't suddenly turn liquid at room temperature? That about sums up the state of software development and bug discovery.

Re:Oh, this sounds like a good idea...

[Anonymous Coward]

but if the bank could demonstrate that it followed avery step without failing any of the certified process, then the blame would be on the certification authority - if the bridge of your example was built using a low quality concrete and falls, (an illegally low quality of concrete) then the inspector which allowed for that concrete to be used should be liable for the bridge fall.

Re:Oh, this sounds like a good idea...

[ArsenneLupin]

How do you guarantee that steel is the best material or that the iron won't suddenly turn liquid at room temperature?

Better analogy would be, how do you guarantee carbonated steel doesn't turn brittle in icy waters or how do you guarantee that the wind doesn't induce fatal vibrations matching the resonant frequency of the bridge.

Indeed, bugs do exist at the time of inspection, they are just not (yet) known. No change of laws of physics is required, only discovery of yet unknown (or underestimated) effects.

Re:Oh, this sounds like a good idea...

[Tom]

If they win this lawsuit, they're setting a dangerous precedent - anyone who at any stage has certified a system as secure becomes responsible for its ongoing security, and can potentially be held liable for stupid user errors by users of that system.

Contrary to the precedent that no matter how much you fuck up, and no matter how blatantly false your audit report is, you're not responsible for anything, including not finding problems that are there when your whole job justification is that you're there to find these problems?

Stop worrying about the poor little techie. We're talking commercial enterprises here. The immediate effect will be that auditing companies take out insurances to cover this risk, and the price of audits goes up a little. However, the secondary effect will be that audits do, in fact, improve, because the premiums on your insurance depend on how often you fuck up and the insurance company has to pay for it.

Re:Oh, this sounds like a good idea...

[Rogerborg]

If they win this lawsuit, they're setting a dangerous precedent

How so? The principle seems clear enough that any audit, in any industry, is only a snapshot; why would you think a court would change that principle in this case?

The article indicates that the system wasn't CISP compliant at the time of the breach, but presumably Merrick can only prevail if they can show that the non-compliant that allowed the breach was also in place at the time of the audit. Do you think otherwise? If so, what leads you the conclusion that the sky is about to fall?

Re:Oh, this sounds like a good idea...

[Smidge204]

So in other words, if the bank can demonstrate that the cert authority didn't do its job properly, the cert auth can be held liable?

Sounds about right to me.

I'd like to see the certs creep up the line of development. Software used for high security applications should be certified at the developer level, and the installation and implementation of that software should be certified at the implementation level.

To continue the bridge analogy: The contractor needs to be licensed and insured, just as the inspector needs to make sure the materials and methods used are up to spec. Are developers held responsible for the quality of their products?
=Smidge=

What about the Dufus?

[siloko]

Well much as I like people to be held responsible for the quality of their work I think it is a bit much to expect technology certification experts to be held responsible for the dufus who puts his username and password on a PostIt stuck to his monitor . . .

Kind of.

[Renraku]

If an inspector inspects and then signs off on an elevator, and the elevator subsequently catastrophically fails due to some reason the inspector should have caught, the inspector can be held liable, unless they can show that his inspection was somehow tampered with. Like perhaps the safety interlocks were just for show and didn't have any real parts inside of them.

Auditors should be held to the same standard, and given the same rights to defend themselves.

I don't want to sound harsh, but considering people pay auditors to do a job, if the job isn't done right, they need to suffer the consequences.

Re:Kind of.

[wirelessbuzzers]

I agree, but it's hard to say what standard auditors should be held to. Often, computer security audits are just surface level checks: they check your design docs and your testing methodology. And this is fine, but you get what you pay for. If a bug slips through your tests, or worse if you don't actually implement your design docs or tests, the auditors obviously shouldn't be liable. On the other hand, if there's a flaw that the auditors "should" have caught, and they don't, they should be liable at least to some degree.

The difficulty is that full, in-depth code audits are very, very hard. Consider the Linux kernel or OpenSSL: even after 16 years of "many eyes" treatment by engineers and security researchers across the world, serious bugs keep showing up. As a result, the fact that the auditor missed something doesn't mean much, and it's not clear that a court will be able to decide whether the auditor "should" have caught it.

I wonder if the same problem is present in other industries.

Re:Kind of.

[Rosco P. Coltrane]

You're correct that, if an elevator cable is frayed and the auditor missed it, he should be sued. However, audits aren't a way for businesses to shift the blame onto the auditor: they're a way for honest businesses to confirm that everybody (employees and contractors) and everything is in order at a certain point in time. If the auditor finds something that isn't right, his job is to inform his client, and perhaps propose remedies, but that's all. It's the business' job to implement the remedies. What I mean is, audits are a tools *for the client* to help do things right, that's all.

For instance, I once subcontracted in a company that used all manners of cracked software. A day or two before the IT audit was due, the manager used to go around telling employees to uninstall anything shady and put away copied CDs. The auditor would come, say everything was good, and the day after, all the cracked software were reinstalled. Is this the auditor's fault? The problem in this case is that the company needed the audit to be this-or-that-certified, in order to work for a certain customer. They didn't see the audit as a tool to help them do business better, but as an annoyance that could prevent them to do IT on the cheap.

Re:Kind of.

[Xest]

The problem is that auditors only check something at that point in time. They can't check that things are correct on an ongoing basis and they can't help it if what they're checking against isn't foolproof.

I used to support IT in schools, and was sent on a PAT testing (http://www.pat-testing.info/) course so that I could PAT test equipment in schools. One thing that was made clear on the course was that if we are not willing to do PAT testing we do not have to even if our employer tells us to. Why? Because if you sign off a piece of electrical equipment as safe and someone injures themselves because it wasn't safe a day later you could be liable - that sounds fair enough at first read through, but what if it really was safe when tested but something happened after testing, before the incident that led to it becoming unsafe? How can you as an tester foresee that? I actually refused to do PAT testing because of this, I simply was not willing to sign myself as liable for something I could not control.

Furthermore, many auditors for example, security auditors can check to ensure a company is complying to security policies, but what if those policies are flawed and a breach occurs because of that? The auditor was paid to ensure policies were followed, and it is the company that is paying for that who is at fault IMO if the policy wasn't enough. Say an IT security policy states that all security patches should be applied immediately, that's great, a security auditor could check that, but what if then there's a breach using a vulnerability for which there was no patch? Is it the auditors fault?

To me it's the company's fault again, the real problem is this, companies don't want to spend time and money on things they see no instant benefit from such as following security policies and procedures. They do the bare minimum they can and comply with the policies and procedures they have to - knowing full well that these policies and procedures are the bare minimum and insufficient for real security and good practice. There's always more that can be done, allowing them to shift the blame just means they'll struggle to find auditors.

Auditors do what auditors are supposed to do, if auditors do their job wrong then sure they should be liable, but I do not see how you can make them liable for something outside their remit. If you pay someone for a full security audit it's one thing, if however you pay them to ensure you're BS7799 compliant and you don't do anything over and above that but suffer a breach as a result of the fact there are things you can do over and above BS7799 then it's your companies fault.

The answer has to come down to the auditor's role, and if the auditor has audited what he's supposed to he should not be at fault. It is only when the auditor has accepted to do an audit and signed it off and that his audit was found to be at fault that he should be liable. In the example of the lift you state though, there is no way that we can know if the auditor was at fault, if he tested it and it really was safe, how could he be at fault if say over night a minor earthquake occured making the lift not safe? What if because of the nature of it he can't prove that it wasn't like that when he tested it? Should he be jailed for manslaughter? When he did nothing wrong at all, should he even have to suffer having his name dragged through the mud, possibly being suspended from work/losing his job in the process until he's finally found not guilty even though his life is wrecked anyway?

Companies should be held liable anyway, if a company gets screwed by a bad auditor it should be on the company to prove the audit itself was faulty. In other words, let's stick to innocent until proven guilty. If a company feels the auditor is guilty, let them prove it, not vice versa.

Re:Kind of.

[Tom]

The problem is that auditors only check something at that point in time. They can't check that things are correct on an ongoing basis and they can't help it if what they're checking against isn't foolproof.

The elevator guy has the same problem and yet it works in real life.

That is because in any real life situation, tests are indeed done repeatedly, such as every quarter, every month - or if they are really important, every day or every event. No plane in the western world takes off without the pilot and co-pilot having run through a standardized checklist first.

"But things can change" is a pretty bad excuse. Like the elevator (where wear and tear change the physics constantly), your system has to be resilient enough to withstand normal changes (e.g. wear and tear, different weights, etc.) at least until the next check. Unauthorized changes have to be hard to make unintentionally (that's why there's no "cut the cable" button inside the elevator).

It really isn't that hard. It works in thousands of areas, many of whom are non-trivial and technically complex (e.g. airplanes). But for some reason, we think it's impossible to do it in auditing and software?

Liable for what, exactly?

[getuid()]

Should the auditor be liable for mis-certification? Or for the (correctly) certified system not withstanding attacks?

I think people should *very* hard try to distinguish between the two scenarios:

1) An auditor certifies a system as XY-compliant as of [insert date here]. However, it can be demonstrated that the system was *not* XY-compliant at that date.

2) An auditor certifies a system as XY-compliant as of [insert date here]. However, at a later date, the system breaks for some reason. It can be proven that the system was XY-compliant, but for some reason (stupid user interaction?) is not anymore. Or, even better: it can be proven that the system *still* is XY-compliant, but the XY-standard is unfit to defend [insert attack here].

I think in case (1) the auditor should be held liable, since he obviously certified something that didn't meet the promised standards. However, in case of (2), not the auditor is to blame. If the system breaks despite of the certification, then it's not the auditor's fault -- it's how things work, and making a scapegoat out of the auditor is not going to do anybody any good. Even worse, if the system fails to meet standard XY because a stupid user (or admin, for that matter) interaction *after* the certification, then there's no way an auditor could have prevented that -- it's either the user/admin's fault for interfering with a certified system, or the standard's fault for not defining what a user/admin is allowed to do with the system without interfering with its certified qualities.

Wow

[Peregr1n]

The big banks really are intent on shooting themselves in the foot. If they hold the auditor liable for security breaches, nobody else will be willing to offer certification services for PCI-DSS. And considering that it's the banks who desperately want everyone to be PCI-DSS compliant (does anybody other than the banks get any benefit from it? Really?), that is particularly stupid.
It's hard enough achieving compliancy as it is - whenever we get near to completing the questionnaire, they change all the questions!

In PCI the auditor does not certify

[hugetoon]

After conducting an audit of a Merchant et a PSP (payement service provider), a QSA (qualified security assesor) issues a ROC (report on compliance to PCI-DSS) that is submitted du issuers (VISA, Mastercard, Amex, JCB and Discover).

Then the issuers certify the auditee.

An individual can not be a QSA by itself, it has to work in an organization that is qualified as well. Among other things a QSA organization has to provision a HUGE amount of cash in case it is found liable of having unduly declared an auditee compliant.

When a breach occurs, there is an investigation and eventually it is found that the ROC was not accurate by the time of the audit in such case the QSA organization and the QSA individual are in trouble.

BTW a certification is only for one year.

Now the case is not about PCI-DSS but "Cardholder Information Security Program" (CISP) and the breach happened in 2005.
Therefore I think the outcome would not have much impact on PCI program where liabilities are well defined.

What is a certification worth?

[bradley13]

The question is: does a certification have a value, or not?

Consider an example in a different area: accounting. At the end of the year, a public corporation must have its accounts certified by an auditor. The audit essentially states that the accounts are an accurate reflection of the company's financial state - that the accountants haven't "disappeared" a few million dollars into their private accounts, or whatever.

If the accounts turn out to be fraudulent, the auditors have failed - and it is entirely correct to sue them.

Back to IT certifications: if the audit missed something, then it is entirely appropriate to sue the auditors. If the security breach was not due to problems the auditors should have caught (inside job, violation of established procedures, etc.), then the auditors should not be liable.

Consider what happens if you do not hold the auditors liable: a very current example from the financial world. The ratings agencies said that derivatives based on sub-prime mortgages were top-quality, low risk investments. Screwing up a rating costs them nothing, so they gave in to political pressure and rated these derivatives too high. Had they been liable for the consequences of their ratings, they would have done a better job. At least, one would like to think so - sadly, there is no way to go back and test this hypothesis...