Slashdot Mirror
Should Auditors Be Liable For Certifications?
dasButcher writes "Enterprises and mid-size business rely on auditors and service providers to certify their systems as compliant with such security regs and standards as PCI-DSS or SOX. But, as Larry Walsh speculates, a lawsuit filed by a bank against an auditor/managed service provider could change that. The bank wants to hold the auditor liable for a breach at its credit card processor because the auditor certified the processor as PCI compliant. If the bank wins, it could change the standards and liabilities of auditors and service providers in the delivery of security services."
I agree, but it's hard to say what standard auditors should be held to. Often, computer security audits are just surface level checks: they check your design docs and your testing methodology. And this is fine, but you get what you pay for. If a bug slips through your tests, or worse if you don't actually implement your design docs or tests, the auditors obviously shouldn't be liable. On the other hand, if there's a flaw that the auditors "should" have caught, and they don't, they should be liable at least to some degree.
The difficulty is that full, in-depth code audits are very, very hard. Consider the Linux kernel or OpenSSL: even after 16 years of "many eyes" treatment by engineers and security researchers across the world, serious bugs keep showing up. As a result, the fact that the auditor missed something doesn't mean much, and it's not clear that a court will be able to decide whether the auditor "should" have caught it.
I wonder if the same problem is present in other industries.
I hereby place the above post in the public domain.
Should the auditor be liable for mis-certification? Or for the (correctly) certified system not withstanding attacks?
I think people should *very* hard try to distinguish between the two scenarios:
1) An auditor certifies a system as XY-compliant as of [insert date here]. However, it can be demonstrated that the system was *not* XY-compliant at that date.
2) An auditor certifies a system as XY-compliant as of [insert date here]. However, at a later date, the system breaks for some reason. It can be proven that the system was XY-compliant, but for some reason (stupid user interaction?) is not anymore. Or, even better: it can be proven that the system *still* is XY-compliant, but the XY-standard is unfit to defend [insert attack here].
I think in case (1) the auditor should be held liable, since he obviously certified something that didn't meet the promised standards. However, in case of (2), not the auditor is to blame. If the system breaks despite of the certification, then it's not the auditor's fault -- it's how things work, and making a scapegoat out of the auditor is not going to do anybody any good. Even worse, if the system fails to meet standard XY because a stupid user (or admin, for that matter) interaction *after* the certification, then there's no way an auditor could have prevented that -- it's either the user/admin's fault for interfering with a certified system, or the standard's fault for not defining what a user/admin is allowed to do with the system without interfering with its certified qualities.
but if the bank could demonstrate that it followed avery step without failing any of the certified process, then the blame would be on the certification authority - if the bridge of your example was built using a low quality concrete and falls, (an illegally low quality of concrete) then the inspector which allowed for that concrete to be used should be liable for the bridge fall.
The problem is that auditors only check something at that point in time. They can't check that things are correct on an ongoing basis and they can't help it if what they're checking against isn't foolproof.
I used to support IT in schools, and was sent on a PAT testing (http://www.pat-testing.info/) course so that I could PAT test equipment in schools. One thing that was made clear on the course was that if we are not willing to do PAT testing we do not have to even if our employer tells us to. Why? Because if you sign off a piece of electrical equipment as safe and someone injures themselves because it wasn't safe a day later you could be liable - that sounds fair enough at first read through, but what if it really was safe when tested but something happened after testing, before the incident that led to it becoming unsafe? How can you as an tester foresee that? I actually refused to do PAT testing because of this, I simply was not willing to sign myself as liable for something I could not control.
Furthermore, many auditors for example, security auditors can check to ensure a company is complying to security policies, but what if those policies are flawed and a breach occurs because of that? The auditor was paid to ensure policies were followed, and it is the company that is paying for that who is at fault IMO if the policy wasn't enough. Say an IT security policy states that all security patches should be applied immediately, that's great, a security auditor could check that, but what if then there's a breach using a vulnerability for which there was no patch? Is it the auditors fault?
To me it's the company's fault again, the real problem is this, companies don't want to spend time and money on things they see no instant benefit from such as following security policies and procedures. They do the bare minimum they can and comply with the policies and procedures they have to - knowing full well that these policies and procedures are the bare minimum and insufficient for real security and good practice. There's always more that can be done, allowing them to shift the blame just means they'll struggle to find auditors.
Auditors do what auditors are supposed to do, if auditors do their job wrong then sure they should be liable, but I do not see how you can make them liable for something outside their remit. If you pay someone for a full security audit it's one thing, if however you pay them to ensure you're BS7799 compliant and you don't do anything over and above that but suffer a breach as a result of the fact there are things you can do over and above BS7799 then it's your companies fault.
The answer has to come down to the auditor's role, and if the auditor has audited what he's supposed to he should not be at fault. It is only when the auditor has accepted to do an audit and signed it off and that his audit was found to be at fault that he should be liable. In the example of the lift you state though, there is no way that we can know if the auditor was at fault, if he tested it and it really was safe, how could he be at fault if say over night a minor earthquake occured making the lift not safe? What if because of the nature of it he can't prove that it wasn't like that when he tested it? Should he be jailed for manslaughter? When he did nothing wrong at all, should he even have to suffer having his name dragged through the mud, possibly being suspended from work/losing his job in the process until he's finally found not guilty even though his life is wrecked anyway?
Companies should be held liable anyway, if a company gets screwed by a bad auditor it should be on the company to prove the audit itself was faulty. In other words, let's stick to innocent until proven guilty. If a company feels the auditor is guilty, let them prove it, not vice versa.
How do you guarantee that steel is the best material or that the iron won't suddenly turn liquid at room temperature?
Better analogy would be, how do you guarantee carbonated steel doesn't turn brittle in icy waters or how do you guarantee that the wind doesn't induce fatal vibrations matching the resonant frequency of the bridge.
Indeed, bugs do exist at the time of inspection, they are just not (yet) known. No change of laws of physics is required, only discovery of yet unknown (or underestimated) effects.
If they win this lawsuit, they're setting a dangerous precedent - anyone who at any stage has certified a system as secure becomes responsible for its ongoing security, and can potentially be held liable for stupid user errors by users of that system.
Contrary to the precedent that no matter how much you fuck up, and no matter how blatantly false your audit report is, you're not responsible for anything, including not finding problems that are there when your whole job justification is that you're there to find these problems?
Stop worrying about the poor little techie. We're talking commercial enterprises here. The immediate effect will be that auditing companies take out insurances to cover this risk, and the price of audits goes up a little. However, the secondary effect will be that audits do, in fact, improve, because the premiums on your insurance depend on how often you fuck up and the insurance company has to pay for it.
Assorted stuff I do sometimes: Lemuria.org
After conducting an audit of a Merchant et a PSP (payement service provider), a QSA (qualified security assesor) issues a ROC (report on compliance to PCI-DSS) that is submitted du issuers (VISA, Mastercard, Amex, JCB and Discover).
Then the issuers certify the auditee.
An individual can not be a QSA by itself, it has to work in an organization that is qualified as well. Among other things a QSA organization has to provision a HUGE amount of cash in case it is found liable of having unduly declared an auditee compliant.
When a breach occurs, there is an investigation and eventually it is found that the ROC was not accurate by the time of the audit in such case the QSA organization and the QSA individual are in trouble.
BTW a certification is only for one year.
Now the case is not about PCI-DSS but "Cardholder Information Security Program" (CISP) and the breach happened in 2005.
Therefore I think the outcome would not have much impact on PCI program where liabilities are well defined.
The question is: does a certification have a value, or not?
Consider an example in a different area: accounting. At the end of the year, a public corporation must have its accounts certified by an auditor. The audit essentially states that the accounts are an accurate reflection of the company's financial state - that the accountants haven't "disappeared" a few million dollars into their private accounts, or whatever.
If the accounts turn out to be fraudulent, the auditors have failed - and it is entirely correct to sue them.
Back to IT certifications: if the audit missed something, then it is entirely appropriate to sue the auditors. If the security breach was not due to problems the auditors should have caught (inside job, violation of established procedures, etc.), then the auditors should not be liable.
Consider what happens if you do not hold the auditors liable: a very current example from the financial world. The ratings agencies said that derivatives based on sub-prime mortgages were top-quality, low risk investments. Screwing up a rating costs them nothing, so they gave in to political pressure and rated these derivatives too high. Had they been liable for the consequences of their ratings, they would have done a better job. At least, one would like to think so - sadly, there is no way to go back and test this hypothesis...
Enjoy life! This is not a dress rehearsal.