Slashdot Mirror


Should Auditors Be Liable For Certifications?

dasButcher writes "Enterprises and mid-size business rely on auditors and service providers to certify their systems as compliant with such security regs and standards as PCI-DSS or SOX. But, as Larry Walsh speculates, a lawsuit filed by a bank against an auditor/managed service provider could change that. The bank wants to hold the auditor liable for a breach at its credit card processor because the auditor certified the processor as PCI compliant. If the bank wins, it could change the standards and liabilities of auditors and service providers in the delivery of security services."

209 comments

  1. Oh, this sounds like a good idea... by fractoid · · Score: 4, Insightful
    TFA makes a very good point:

    What will be interesting about this lawsuit is how the court assigns responsibility for a breach at a certified business. Audits, by their very nature, are point-in-time or snapshot checks. They cannot account for the dynamic variables of business and IT operations that may weaken security over the long-haul.

    If they win this lawsuit, they're setting a dangerous precedent - anyone who at any stage has certified a system as secure becomes responsible for its ongoing security, and can potentially be held liable for stupid user errors by users of that system.

    --
    Rampant carbon sequestration destroyed the Dinosaurs' tropical paradise. I'm here to help repair the damage.
    1. Re:Oh, this sounds like a good idea... by Renraku · · Score: 4, Interesting

      Inspectors of things like elevators are not responsible if their target checked out at the time of inspection, and later failed. For example, you could sign off on the construction of a bridge or an installation of an elevator because everything looked good, but when the bridge company doesn't maintain the bridge properly or the elevator company fails to do the same, the inspector is not held liable, even though they were certified as good.

      Auditing a network should be the same way. Of course, an auditor should NOT be held responsible for undiscovered bugs or holes in software. Instead, their job should focus on general security. It would be like a bridge inspector trying to certify a bridge when the gravitational constant of the universe were in a state of flux. How do you guarantee that steel is the best material or that the iron won't suddenly turn liquid at room temperature? That about sums up the state of software development and bug discovery.

      --
      Job? I don't have time to get a job! Who will sit around and bitch about being broke and unemployed then?
    2. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 5, Interesting

      but if the bank could demonstrate that it followed avery step without failing any of the certified process, then the blame would be on the certification authority - if the bridge of your example was built using a low quality concrete and falls, (an illegally low quality of concrete) then the inspector which allowed for that concrete to be used should be liable for the bridge fall.

    3. Re:Oh, this sounds like a good idea... by ArsenneLupin · · Score: 5, Insightful

      How do you guarantee that steel is the best material or that the iron won't suddenly turn liquid at room temperature?

      Better analogy would be, how do you guarantee carbonated steel doesn't turn brittle in icy waters or how do you guarantee that the wind doesn't induce fatal vibrations matching the resonant frequency of the bridge.

      Indeed, bugs do exist at the time of inspection, they are just not (yet) known. No change of laws of physics is required, only discovery of yet unknown (or underestimated) effects.

    4. Re:Oh, this sounds like a good idea... by noundi · · Score: 3, Insightful

      I highly doubt that's even the case. The bank would probably have to prove that the breach could have taken place even at the time of auditing, not after, due to obvious reasons anyone can imagine. If they manage do to so the suit should be perfectly valid.

      --
      I am the lawn!
    5. Re:Oh, this sounds like a good idea... by Tom · · Score: 5, Insightful

      If they win this lawsuit, they're setting a dangerous precedent - anyone who at any stage has certified a system as secure becomes responsible for its ongoing security, and can potentially be held liable for stupid user errors by users of that system.

      Contrary to the precedent that no matter how much you fuck up, and no matter how blatantly false your audit report is, you're not responsible for anything, including not finding problems that are there when your whole job justification is that you're there to find these problems?

      Stop worrying about the poor little techie. We're talking commercial enterprises here. The immediate effect will be that auditing companies take out insurances to cover this risk, and the price of audits goes up a little. However, the secondary effect will be that audits do, in fact, improve, because the premiums on your insurance depend on how often you fuck up and the insurance company has to pay for it.

      --
      Assorted stuff I do sometimes: Lemuria.org
    6. Re:Oh, this sounds like a good idea... by Rogerborg · · Score: 4, Informative

      If they win this lawsuit, they're setting a dangerous precedent

      How so? The principle seems clear enough that any audit, in any industry, is only a snapshot; why would you think a court would change that principle in this case?

      The article indicates that the system wasn't CISP compliant at the time of the breach, but presumably Merrick can only prevail if they can show that the non-compliant that allowed the breach was also in place at the time of the audit. Do you think otherwise? If so, what leads you the conclusion that the sky is about to fall?

      --
      If you were blocking sigs, you wouldn't have to read this.
    7. Re:Oh, this sounds like a good idea... by IBBoard · · Score: 1

      Ditto for MOT tests in the UK (from what I've heard from my sister-in-law in the US the Americans don't have a similar "road-worthiness test"). The MOT says your car is safe to go on the road, doesn't have emissions that are too high, etc, but it also says that it is a one-off test and that it doesn't make any guarantee of on-going quality. Just because the garage checked the car over on Monday and thought it was okay doesn't mean it is okay on Friday after the driver has run it over large curbs or high-speed over extremely rough and rocky terrain.

    8. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      Good Auditors leave the back door open, say yes, but have a mile of qualifications. After all who reads page 246?
      like based on what was presented/asserted
      That that was accurate and checked my the MD
      and that things could change if the client failed to warn or notify us of xxxxx
      all care no responsibility
      in the event of a court case, maximum damages $100 dollars.

      For years they have been poking their beaks into areas they no nothing about, but do have a pretty good checklist.
      Think Enron, think Barrings, think big banks rescued? by the guvment.

    9. Re:Oh, this sounds like a good idea... by asdf7890 · · Score: 2, Insightful

      If they win this lawsuit, they're setting a dangerous precedent - anyone who at any stage has certified a system as secure becomes responsible for its ongoing security, and can potentially be held liable for stupid user errors by users of that system.

      IMO it depends on where the fault lies.

      If the fault that allowed the problem is a property of the system that an auditor or penetration tester could be reasonably expected to have picked up on (such as password complexity and cycling rules not being present or not being correctly enforced) then maybe the case is valid.

      If on the other hand the problem is outside the system that was audited (i.e. the breach was due to a user having stored/transmitted a copy of their credentials insecurely, or due to users/admins not being adequately trained, or due (or due in part) to software/configuration/network changes made after the audit was complete) then there is no way the auditor should be held responsible.

      In reality all that will happen which-ever way this case goes is that there will be chunks of new boiler-plate exceptions text in future relevant contracts or the auditors will charge companies more in exchange for underwriting the extra risk. At work we are currently playing piggy-in-the-middle with the agreements for penetrations testing a new system we are building for a client and there is a lot of contracts work that goes on sorting out who is allowed to do what and who (us, the DC and equipment provider, the client and the 3rd party testers) is responsible for what now and going forward - this case will do no more in the long run than to add extra items to those lists (an increase the relevant consultation fees too, of course).

    10. Re:Oh, this sounds like a good idea... by Chris+Mattern · · Score: 2, Informative

      from what I've heard from my sister-in-law in the US the Americans don't have a similar "road-worthiness test"

      It's up to the individual states, but most states have them. Here in Virginia, I have to get my car safety inspected once a year (and carry an inspection sticker on my windshield) and emissions tested once every two years (or they won't let me renew the car's registration).

    11. Re:Oh, this sounds like a good idea... by Ihlosi · · Score: 3, Informative
      how do you guarantee that the wind doesn't induce fatal vibrations matching the resonant frequency of the bridge.

      Quote from the linked page:

      "In the case of the Tacoma Narrows Bridge, there was no resonance."

      That bridge came down due to a profoundly nonlinear positive feedback effect (the deformation caused by the wind increased the area of attack, which lead to more deformation, etc), not due to the bridge resonating.

    12. Re:Oh, this sounds like a good idea... by Smidge204 · · Score: 4, Interesting

      So in other words, if the bank can demonstrate that the cert authority didn't do its job properly, the cert auth can be held liable?

      Sounds about right to me.

      I'd like to see the certs creep up the line of development. Software used for high security applications should be certified at the developer level, and the installation and implementation of that software should be certified at the implementation level.

      To continue the bridge analogy: The contractor needs to be licensed and insured, just as the inspector needs to make sure the materials and methods used are up to spec. Are developers held responsible for the quality of their products?
      =Smidge=

    13. Re:Oh, this sounds like a good idea... by Z00L00K · · Score: 1

      This is an interesting case to have.

      • If the auditor certifies a system according to current regulations and the system later fails. Is that the fault of the auditor or the regulations?
      • System changes can render the certification invalid and then the system has to be revalidated.
      • New threats and hacking methods appears all the time, so even current regulations may be outdated.
      • You shall never certify your own system, always bring an outside certification agency. Then it's up to you to take action and responsibility.
      • Always expect at least one security measure to fail. This means that you shall never rely on a single protection as a pin code or a password for critical systems.
      • It's your system, so you should have the ultimate responsibility.
      • But then - who certifies the auditors? Do they have an up to date certification? Don't let this stop you - even auditors without certifications can be really good.
      • Every component in a system may check out really well, but when they interact you may have a hole as big as Grand Canyon.
      • Third-party problems like hijacked certificates can cause a major headache.

      So I would rather say that if an auditor is in the auditing report showing incompetence and negligence - then the auditor is a valid target, but if the documentation is covering the system well then the customer shouldn't be able to complain. And it's also hard for an auditor to be able to verify every aspect of a system without an extended study and analysis of the code, possible backdoors and system design.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    14. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      I'm not sure how much you know about PCI-DSS, but I think it would be a bigger change than you think - for small (non-techie) companies anyway.

      At present it is very easy to get your server certified. Basically be running up to date software (or if this is too hard, configure your box to report it is running up to date software - if you can't even manage this, call them up and tell them that you are running up to date software and their scan must be wrong. Really!), and then deal with any SQL injection or XSS vulnerabilities that they turned up. Then pay them whatever money they ask for and all done, you are PCI compliant (regarding the webserver anyway).

      Now I dunno about you, but if I was an insurance company, I'd be charging a HEFTY premium if I was to insure the auditing company against problems resulting from this procedure.

      As far as I can see, a change like this would be so large as to pretty much make it impossible for a small company to handle cards on its own servers. Which would be a good thing all round IMO.

    15. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      If they win this lawsuit, they're setting a dangerous precedent - anyone who at any stage has certified a system as secure becomes responsible for its ongoing security, and can potentially be held liable for stupid user errors by users of that system.

      Contrary to the precedent that no matter how much you fuck up, and no matter how blatantly false your audit report is, you're not responsible for anything, including not finding problems that are there when your whole job justification is that you're there to find these problems?

      Stop worrying about the poor little techie. We're talking commercial enterprises here. The immediate effect will be that auditing companies take out insurances to cover this risk, and the price of audits goes up a little. However, the secondary effect will be that audits do, in fact, improve, because the premiums on your insurance depend on how often you fuck up and the insurance company has to pay for it.

      Tertiary effect: customers, auditors, and the insurance companies you mentioned turn on vendors (*cough* Microsoft *cough*) that produce products with crappy security and hold THEM liable.

    16. Re:Oh, this sounds like a good idea... by Runaway1956 · · Score: 1, Informative

      "Audits, by their very nature, are point-in-time or snapshot checks."

      8 years military service here. Security was 24/7 plus when I was in uniform. There was no "snapshot" of security, because everyone was trained from day one to understand that a moment in time is meaningless.

      I have always laughed at the concept of "security" in most of the civilian world. Seldom have I been in any civil institution where real security measures were in place, and enforced - be that physical or electronic. Oh, there ARE places that are secure, but most banks are a sad, sad joke when it comes to security.

      Security providers especially should be liable. They have a contract to provide security, they can't come around every few weeks and check on how things are going.

      An auditor has less responsibility than a provider, but even so, he should realize that a "snapshot" is only a fleeting moment in time. If he doesn't understand that he needs to spend DAYS on site to understand not only how things are SUPPOSED to work, but how they DO work, then he isn't competent to pass himself off as a security auditor.

      To be perfectly honest, it all comes back to the management, though. There are precious few managers who will part with the money necessary to hire competent security, or to enforce strict compliance with real security measures. Again, that is true of physical security, AND electronic security. The day that someone such as a bank manager pulls his head out of his arse, and realizes that security is costly, the day that he PAYS FOR competent security personnel, THEN his bank will become secure.

      It's a good thing to begin to hold these auditors and providers accountable. At least 90% of them are lax, and at least 70% of them are incompetent. A little liability will teach them to learn their jobs, then to perform their jobs properly. It will cost, but everyone will benefit, in the end.

      Well, everyone will benefit except those who are exploiting the present lack of security.

      --
      "Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
    17. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 3, Insightful

      Sounds like you're assuming that being PCI compliant is in fact the same thing as being 100% secure, which is retarded. They were supposed to make sure the servers were PCI compliant... that is all.

    18. Re:Oh, this sounds like a good idea... by hairyfeet · · Score: 2, Interesting

      I can see the possibility of time playing a factor in this as well. To keep with the bridge analogy, if I get a bridge certified safe, and it falls down around my ankles in 3 weeks even though I did everything the certifying contractor told me to do, I can see where there would be a lawsuit.

      To apply that to this case, if the auditor certified them to be PCI compliant and they followed all the rules and it fell down around them in two weeks I think it'd be safe to say the auditor may have missed something and be liable for it. So while I can't see the auditors having to worry long term, as there are just too many variables that they aren't in control of, i could see them being on the hook if a company can show they followed the rules and it fell apart in a short period of time, say six months. Because that to me would make me wonder if the auditor was overworked and missed something small. And as all of us tech guys know it is the small things you overlook that come back to bite you in the ass.

      --
      ACs don't waste your time replying, your posts are never seen by me.
    19. Re:Oh, this sounds like a good idea... by Opportunist · · Score: 2, Interesting

      Security is a 24/7 process. Audits are snapshots thereof.

      There are quite a few companies that dread and fear their 9001 or (even more) 27001 renewals because they are "so much work". Yes they are, if you're not sticking to the certification requirements (which you technically have to, after all that's what the sheet of paper that you get certifies).

      Every time a company moans about "certification work", I question their certification worthyness.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    20. Re:Oh, this sounds like a good idea... by azadrozny · · Score: 1

      Comparing mechanical devices like a car, that have parts that wear down to a network which is not susceptible to the same pressures is not completely fair. If my mechanic certifies that my car passes the state safety inspection (which we do have in the US) on Monday, and I suffer a catastrophic failure of one of the inspected parts on Friday, then I might have a case. In six months, I probably don't.

      I see inspecting/certifying a network as being a little different. If I certify that your network meets a certain standard, protecting you from X, Y, and Z types of attacks, then baring a change to the network's configuration (thereby voiding the certification) you should always be protected. If in the future you are attacked using one of these methods, then shame on me for not being thorough. However, this does not let you off the hook for protecting yourself against new types of attacks.

    21. Re:Oh, this sounds like a good idea... by mysidia · · Score: 1

      And if the defect is something the inspector can't see because it's obscured inside the bridge structure?

      E.g. they used good quality concrete on top, the low quality stuff was at the bottom/just under, under it, and just a small percentage of the concrete used (but in retrospect still enough and in a bad place allowing it to collapse).

    22. Re:Oh, this sounds like a good idea... by mysidia · · Score: 2, Informative

      Except those two specific conditions, and in theory (how to prevent them) are well-known.

      The unknown bugs software has are new cases entirely that cannot be examined a priori like a bridge's aerodynamics can.

    23. Re:Oh, this sounds like a good idea... by Nikker · · Score: 2, Informative

      PCI compliance is mostly about network security and infrastructure, such as ensuring networks that service secured endpoints are isolated from networks that aren't. The auditor is really only there to attempt to mitigate and isolate known security issues that most shops don't bother to take too seriously. By starting this buck passing all you are really doing is starting a new age of insurance that you will need to take to cover the possible fraud that can take place rather than working with the banks to just keep it to a minimum and deal with the one offs. I do believe that if an auditor checks out a network / system and approves a network / protocol that is insecure by their own standards then of course it is the fault of the auditor and the responsibility of the auditors company to clean up the mess. As many are alluding to as far as OS exploits and the like no one is really able to prevent or anticipate all these possibilities and those are just the "breaks".

      As I said before looking for a fall guy (especially when both parties are financially powerful) will never resolve anything rather than finding a way to screw the business running the system that was audited. This will likely be too much of a liability for many to handle and will rather come out of your pocket in other ways. If you think any financial type business will actually take responsibility on paper or other wise for anything then you are way to new to this game to be making decisions like this.

      --
      A loop, by its nature, continues. If that didn't make sense, start reading this sentence again.
    24. Re:Oh, this sounds like a good idea... by Chaos+Incarnate · · Score: 1

      But it's not like a change to the network's configurationis terribly difficult to cause. One employee downloading & running malware on their desktop constitutes a change in its configuration.

      --
      Benford's Corollary to Clarke's Law: "Any technology distinguishable from magic is insufficiently advanced."
    25. Re:Oh, this sounds like a good idea... by Tom · · Score: 1

      Well, as I see it, if every alternative but the first one would disqualify you, i.e. running up-to-date software is your only choice, then that alone would do a lot to improve security.

      And if that's too big of a change for some companies, then I say: "Good riddance!"

      --
      Assorted stuff I do sometimes: Lemuria.org
    26. Re:Oh, this sounds like a good idea... by itsdapead · · Score: 1

      If they win this lawsuit, they're setting a dangerous precedent - anyone who at any stage has certified a system as secure becomes responsible for its ongoing security

      No, to win, they will presumably have to prove that their systems weren't compliant at the time of the audit. All the TFA says is that the later investigation showed non-compliance - it gives no indication as to the nature of this problem.

      Say I inspect your security, claiming to be an expert, and a few weeks later you have a breach. If, after the inspection, someone re-set a password to something lame and/or left it on a post-it than don't blame me. If, however, it turns out that your wireless router doesn't support encryption - something that's unlikely to have changed since the inspection - then I haven't done a very good job, so why on earth should I not share the liability?

      It also depends what level of expertise is expected from customer and inspector: in some cases the inspector will just be independently verifying what the customer should already know, in other cases the inspector could reasonably be expected to act as an expert: If I'm designing cellphones then I should have a ruddy good idea of whether my product meets FCC standards; If I run a small business and pay an accountant to check my tax returns then I expect them to know a darn site more about tax law than me.

      --
      In a survey of 100 programmers, 111111 thought that duck-typing was a good idea.
    27. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      it followed avery step without failing any of the certified process, then the blame would be on the certification authority - if the bridge of your example was built using a low quality concrete and falls, (an illegally low quality of concrete) then the inspector which allowed for that concrete to be used should be liable for the bridge fall.

      A certification don't guarantee exactly the same implementation of the process everywhere and it does not guarantee security in a changing world filled with external influences. If the banks implementation followed PCI-DSS and the auditor did its job with a generally accepted level of precision, the damages should be covered by the banks insurance policy.

    28. Re:Oh, this sounds like a good idea... by D3 · · Score: 1

      First, it is way too easy to hide information from the PCI assessors. BTW, they are NOT auditors, they are assessors, there is a big difference. But it is too easy to hide stuff because to really dig into a complex system for every last detail is already cost prohibitive.

      Which brings me to my second point. If liability gets pushed to the assessors (or SOX auditors which are real auditors) then the cost of being assessed/audited are going to skyrocket because they will just pass the cost of liability right back to the company that hired them. The companies being assessed/audited are being held at 'legal gunpoint' to comply and pay whatever cost. Then, the cost of being assessed will be passed on to the consumers or the company will go out of business.

      --
      Do really dense people warp space more than others?
    29. Re:Oh, this sounds like a good idea... by Runaway1956 · · Score: 1

      We seem to have a dysfunctional definition of the word "audit". When the IRC audits an individual, or a company, they look at the overall picture of earnings for the year, or for multiple years. The intent is to itemize EVERYTHING, and to ensure that everything is accounted for. The IRS has a functioning definition of the word "audit". There is little chance of hiding or obscuring anything that has happened in the period of time being audited.

      A security audit should serve much the same purpose. Whether the auditors look at all records for a month, or a year, they should remain onsite for a period of time, that period being sufficient to truly understand how things are SUPPOSED to work, and how things work in reality.

      A working definition of the word "audit" is simply not a momentary snapshot.

      Should I be employed to audit your security, I'll NOT show up at your jobsite for a day, then send you some half-assed report, stating that your recorded procedures look good. That is nothing more than rubber stamping what your management has already decided to implement.

      --
      "Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
    30. Re:Oh, this sounds like a good idea... by bleh-of-the-huns · · Score: 1

      You are correct, the inspector would be liable, not only for the failure of the bridge, but for failing in his duties as the concrete used was inferior... The big difference, its not hard to test concrete, that is one aspect of bridge building... In IT, its completely different, its not tangible (well in some cases like physical security it is), but an inspector or auditor could have done his job perfectly fine, and things still went wrong as IT rules and policies, and configurations, and patches, and things like that change all the time..

      Sure the company could have followed the steps, patching everything that needed it keeping up with security updates etc etc.. well guess what, sometimes those security updates cause problems, sometimes those patches cause problems. Unless your going to pay the auditors to run a compliance check after every change you make, they are not going to be responsible.

      --
      I came, I conquered, I coredumped
    31. Re:Oh, this sounds like a good idea... by evilkasper · · Score: 1

      The thing that really needs to be taken into account is that just because your certified does not guarantee 100% security. The auditor should not be held responsible for this, all they do is check to see if you are compliant with a standard. If they want 100% secure they should unplug it , put it in safe and then drop it into the Marianas Trench.

    32. Re:Oh, this sounds like a good idea... by Fieryphoenix · · Score: 1

      Not to mention, since when does "conforms to a particular security standard" equal "impregnable"?

    33. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      "Inspectors of things like elevators are not responsible if their target checked out at the time of inspection, and later failed."

      yes, but... you can bet the inspector will be haled into court (or a deposition at the least) and have to explain what they did at that inspection and whether they could have overlooked anything. If it were to go to court, ultimately, the inspector will not be liable, but it could cost a substantial amount of their time and attorney's time to make the point.

      A typical low end insurance policy will be several thousand dollars a year for this kind of thing. Look up "professional E&O insurance"

    34. Re:Oh, this sounds like a good idea... by JumpDrive · · Score: 1

      No, in this case concrete does not equate to a complex IT system audit, this is comparing apples and oranges.
      Concrete has certain specifications and requirements and should be checked by the company that sells it before it is sold for quality. If it is found that the concrete was not mixed right, then they can be found liable.

      An audit is usually a spot check of a system. They pick certain areas and randomly check certain things. If they pass those things, lets say it is 10 items out of possibly a hundred, then they get certified. In this case, it may mean that the company didn't look at the fine print, and the auditors will only have to show there audit reports, which should show a good faith effort to check compliance of the standard.

      What the company is probably trying to do is obfuscate the fact that they were not making a good faith effort to meet the standards. Like maybe they laid off there most experienced coders, network admins,.... and generally anybody who knew anything about keeping a system secure.

      All and all, from a computer technologist standpoint it's a win-win situation. If the auditors are found to be at fault- then audit companies will have to hire more experienced people and do more thorough examinations and they will just charge companies more money and the companies will also have to hire better staff with better training to meet the more rigorous examination. If the company loses, then they will be held accountable for the security breach and be liable to for any damages to their customers, which will tell companies that they need to better train their staff and pay to keep more talented people.

      But what will probably happen is that both sides will get more insurance and better lawyers, because it's just those damn courts and judges, and couldn't possibly be due to their HR or business practices.

      I have seen cases where both sides could be held liable. In one case auditors were shown a dummy site which wasn't actually being used, but was set up to meet the requirements of the audit and in another case I have seen where auditors didn't write up a major infraction, my guess is because they thought they might lose the companies business.

    35. Re:Oh, this sounds like a good idea... by Xiterion · · Score: 1

      They're known because those incidents happen. Prior to those, nobody thought to examine that aspect. Things like this happen quite often where an aspect of a design is not investigated because it hasn't caused a problem yet, but still leads to very real weakness in the product. Software gets this more than other, more established fields of engineering because it is such a comparatively young field. We don't have enough experience with it yet to have uncovered the majority of possible failure modes.

    36. Re:Oh, this sounds like a good idea... by david_thornley · · Score: 1

      Those conditions are well-known now. Nobody at the time realized why "Galloping Gertie" was so lively, and nobody had good evidence that it might destroy itself. A bridge inspector, using techniques and knowledge available at the time, would have certified that bridge.

      --
      "When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
    37. Re:Oh, this sounds like a good idea... by JumpDrive · · Score: 3, Funny

      Are developers held responsible for the quality of their products? Yes, Microsoft developers are held responsible for the quality of their products, can't you tell.

    38. Re:Oh, this sounds like a good idea... by Opportunist · · Score: 1

      Should I be employed to audit your security, I'll NOT show up at your jobsite for a day, then send you some half-assed report, stating that your recorded procedures look good. That is nothing more than rubber stamping what your management has already decided to implement.

      Then don't quit your day job and hope to be employed as an auditor. The sad truth is that companies don't want security. They want certificates. And they will go with whoever gives it to them with the least amount of work necessary.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    39. Re:Oh, this sounds like a good idea... by azadrozny · · Score: 1
      You are correct that malware running on the network is a serious threat. The point I was trying to make is that if an auditor certifies that your network is protected from various type of malware attacks, then they could be held liable if you hacked in this manner.

      I will admit that this is a very gray area, but if you offer your services as a network auditor, then expect to be held liable for failing to anticipate common threats. You should not just be auditing a static network at a single point in time, but also the policies and procedures for maintaing the system.

    40. Re:Oh, this sounds like a good idea... by Smidge204 · · Score: 1

      You really don't seem to grasp how this works in other industries.

      (Disclosure: My only assumption here is that the scope of the auditor's responsibility is clearly defined. If it isn't, then the auditor is just asking for a lawsuit.)

      If it can be demonstrated that the cause of failure was outside the scope of the auditor's contract, then he would not be held liable. For example, the auditor would probably not be responsible for the locks on the doors to the data center - so if someone breaks in and steals the servers, it's not his problem.

      If it can be demonstrated that any potential security holes the auditor did not address were unknown at the time, then he is very likely (although perhaps not certainly) off the hook as well. It suspect would depend on a number of factors that a court would have to figure out. As long as the audit can be shown to have been conducted in good faith with proper methods and the best information available at the time, he should be covered. (Again, unless his contract writer phoned it in and scope of responsibility was poorly defined.)

      If, however, the quality of the audit or preventative measures were poor, then the auditor SHOULD be liable. Why should he not be? If you hire an inspector to certify a new bridge, and he does not do his job, then he should be liable for it.

      This is also why I said the chain of liability should extend further up the line. If there is an undiscovered flaw in the operating system that the auditor didn't know about, then why isn't the OS vendor liable for it?

      The way I see it, there is only a very, very narrow set of circumstances where "shit happens" is a legitimate excuse, especially in the digital security field where so few variables are uncontrolled.

      Compared to, say, bridges - where things like weather or geologic activity or deliberate attacks may be approximated or anticipated but never fully planned for, hence safety factors.
      =Smidge=

    41. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      So, basically what you are saying is, an auditor should be able to foresee that a bug in a linux or windows server that has gone undisclosed as a zero day exploit will activate in the near future, and thus have the company institute logical, physical, and administrative controls to ensure that they stay safe ... Impossible, impractical, unrealistic. An audit IS a snapshot in time. Network environments DO NOT REMAIN STATIC, and neither do vulnerabilities. An auditor can not reasonably be expected to be held liable for not foreseeing an exploit that shows up even a month after the audit finished, if there was no reasonable means to suspect that the exploit existed in the first place, nor that existing controls were insufficient to address the threat. The definition of a zero-day exploit is one that takes advantage of a vulnerability that has NO means of patching/mitigation. An auditor simply can't be held accountable for that. Additionally, the moment a company applies a patch to a system on the network, then by definition, that network environment has changed. It is not the auditor's fault if something breaks after that.

      If an auditor was held liable for a company failing to: implement proper code review of in-house apps, review web pages for vulnerabilities, validate SQL servers for possible injection vulnerabilities, validate firewalls for proper rules auditing, determine that system/network administrators aren't using the same password for multiple systems, etc., then there would be no incentive to certify any company without first being properly insured against eventual lawsuits that WILL be filed WHEN a company network gets breached. An auditor can't guarantee that a company WILL do everything correctly, nor can a company guarantee that it WON'T be vulnerable to an unknown/undisclosed/unpatched exploit even in the NEAR future after an audit. All the audit says is that the company network meets the requirements to be certified to a particular standard.

    42. Re:Oh, this sounds like a good idea... by carmaa · · Score: 1

      Jesus, don't we have a car analogy to go with this legal issue? All this talk about bridges are downright confusing...

      --
      From the dark, old days of the Internet when men were men, women were men, and children FBI agents
    43. Re:Oh, this sounds like a good idea... by Runaway1956 · · Score: 1

      That being the case, then the company hiring the auditor has no recourse when the audit proves to have no value whatsoever. And, if this really is the case with the suit being filed, then I certainly hope that they lose their case.

      I mean, FFS, I can type out certificates, print them out, and sell them to you. You can wallpaper your office with said certificates, but they won't stop some script kiddie from stealing what he wants, then wiping your drives.

      --
      "Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
    44. Re:Oh, this sounds like a good idea... by nosferatu1001 · · Score: 1

      IAANJNYA (I am an auditor just not YOUR auditor)

      Audits are point in time and historical, by their nature. PCI-DSS compliance audits require reviewing security over the year, against the requirements, including the security at the time of the audit.

      So no, a proper security audit isnt "on site 1 day, send report on procedures" - an audit looks at:

        - are controls designed? If not are there any control gaps? Are there sufficient mitigating controls?
        - are those controls operating effectively, or have there been exceptions during the year? If so what is the potential / actual impact of those exceptions?

      Unfortunately the GP doesnt appear to understand what an audit is and can do. The clue was your response of "Security is 24/7, audits are point in time" - no matter how long you are on site, the client can drop all controls the day after you leave and you cannot find this out until the next audit.

    45. Re:Oh, this sounds like a good idea... by azadrozny · · Score: 1
      I mangled my thoughs a bit when I said an audit is not a single point in time. It is. My thinking is that auditors should be taking a "wholelistic" approach, helping the organization look into the future, and making sure their procedures will protect them to the extent possible. For example, patch your operating system regurarly. I think this is what the author meant when he said an audit should "focus more on processes rather than implementation."

      The article was light on details, but Merrick Bank hired auditors, Savvis, to certify that were compliant with the CISP standard. If Savvis was negligent, as Merrick charges, and they were not compliant, then why shouldn't they be held liable? If the breach occured via a security hole that the audit should have caught, then I say let the suit go forward.

    46. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      What if the breach had occured prior to the audit? If I am not mistaken, this is the case with CardSystems. However, nothing in the CISP 1.0 audit proceedures directed the auditors to detect an existing breach. It only has them validate that certain systems are in place to reduce the risk of a breach.

    47. Re:Oh, this sounds like a good idea... by mindstrm · · Score: 3, Interesting

      PCI covers more than just servers ---- it covers physical security, staff identification, physical access to paperwork, disposal, data retention, lots of corporate policies.......

    48. Re:Oh, this sounds like a good idea... by Zerth · · Score: 2, Insightful

      PCI is just troweling mortar on a crumbled foundation. Sure, it covers all the really boneheaded stuff, like using decent authentication and applying patches, but there is no part of it that says "don't use badly made(but it is expensive, it must be good) software on a fundamentally broken OS"

    49. Re:Oh, this sounds like a good idea... by ZouPrime · · Score: 2, Informative

      Exactly. This is a very important disctinction that some peoples fail to grasp.

      An auditor basically compare a situation vs a checklist of auditable issues. He's NOT there to find your security vulnerabilities and tell you to fix them. He's there to tell you that you do or don't respect requirement XYZ. If an issue isn't covered by the standard's requirements, well, what can he do? He can always make a formal observation, but that's beyond the scope of his responsabilities.

      Standards such as PCI, SOX, NERC CIPs etc. aren't designed to protect you against all known threats, they are designed around the general, most common, most problematic security issues. A company can pass an audit and still be very insecure.

    50. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      I just wanted to add,

      I've seen a lot of businesses groups chase their tail on security, never really doing a very good job, with other Security Engineers standing around watching them fail and making helpful suggestions, who seem to think that the problems are going to be fixed by small tinkering.

      No, it's much much worse than this. The tip of the iceberg - if you run Windows in your shop - download MBSA (Microsoft Baseline Security Analyzer) - point it at your servers, analyze the obvious things that just jump out to a scripted check - and realize that at many places the number of Windows computers with critical problems can exceed 90%.

    51. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      There is this thing called a "disclaimer". It says that IF you adhere to certain requirements, THEN the guarantee is good.

      There's also another thing called a "post-mortem". When something crashes, forensics experts can usually do a pretty decent job of back-tracking the causes, whether it's an airliner or a credit card system. Which is how the legal people determine whether the disclaimer is valid.

      It's ironic when you can sue and win major amounts for trivial reasons that there should be room for a complete denial of liability by the very people whose job it is to assert the security and/or quality of anything.

      Then again, maybe I expect too much. Everyday Low Prices and gitter-dun-quick have far higher priorities these days than true quality or reliability.

    52. Re:Oh, this sounds like a good idea... by skuzzlebutt · · Score: 1

      That's what happened to heartland...PCI doesn't cover sploitz or whatever it was that they found.

      --
      My debut novel AMITY now available: http://jeremydbrooks.c
    53. Re:Oh, this sounds like a good idea... by Zader · · Score: 1

      In reality, the auditor typically has minimal technical competency, and is running a canned set of tools that throw out so many false positives that the reports are practically worthless -- or if followed to the letter would make a system fail to even perform it's function. Or in some cases even boot. They even may not have the a canned set of tools for the right OS in the first place, making the reports even more useless.

      Universally true? No. But it's been true in my experience dealing with PCI auditors with one of the major credit card processors. The processors are interested in demonstrating compliance, which may or may not have anything to do with real world security or actual deep inspection of the security of the systems.

      And yes, nobody, including the credit card processors, wants to take the blame or the responsibility. IT is overhead to them, which cuts into the bottom line -- therefore there's little to no interest in hiring people qualified (and with sufficient authority) to properly protect the systems in question. Not to mention the infrastructure investment to go with it.

      PCI compliance tends to leave a false sense of security to organizations that don't understand IT in the first place.

    54. Re:Oh, this sounds like a good idea... by mattwarden · · Score: 1

      The auditor's customers will always be liable for the cost, not matter the outcome. The only question is whether they pay the compensation directly to those damaged or via higher prices for audits.

    55. Re:Oh, this sounds like a good idea... by mattwarden · · Score: 1

      I guess I didn't really explain. The reason is audits are usually mandatory, so there is little or no loss of demand due to higher prices.

    56. Re:Oh, this sounds like a good idea... by ToasterMonkey · · Score: 1

      8 years military service here. Security was 24/7 plus when I was in uniform. There was no "snapshot" of security, because everyone was trained from day one to understand that a moment in time is meaningless.

      For security, sure, but we're talking certifications here. There is no such thing as 24/7 certification, they are issued once a year or so, and you can fuck everything up in between.
      Think of all the annual training that you (hopefully, mostly) received. Are you likely to forget how to fire a rifle, tread water, or don a gas mask between qualifications? Probably not, but I'll bet you didn't regularly train in those unless you were infantry. Every so often someone will even fail a semi-annual fitness test, and that "skill" is almost universally maintained year round. Businesses don't do security just for the sake of security, that's not what they're in it for, unlike the military.

    57. Re:Oh, this sounds like a good idea... by mattwarden · · Score: 1

      > However, the secondary effect will be that audits do, in fact, improve, because
      > the premiums on your insurance depend on how often you fuck up and the insurance
      > company has to pay for it.

      You are essentially suggesting that audit quality does not current affect an auditor's demand and the market instead would benefit from direct price impacts of insurance premium changes due to audit quality. Do I restate you correctly?

    58. Re:Oh, this sounds like a good idea... by TemporalBeing · · Score: 1

      If, however, the quality of the audit or preventative measures were poor, then the auditor SHOULD be liable. Why should he not be? If you hire an inspector to certify a new bridge, and he does not do his job, then he should be liable for it.

      In other industries (e.g. the bridge building) it can also mean criminal prosecution of the auditor - for criminal negligence. I would argue the same should apply here.

      --
      Truth is like the sun. You can shut it out for a time, but it ain't goin' away. - Elvis Presley (source: imdb.com)
    59. Re:Oh, this sounds like a good idea... by dave562 · · Score: 1

      They are held responsible for their products, but not in the legal sense of the meaning responsible. They are held responsible in the marketplace. When the quality of their products sucks, they pay for it in market share. Their search technology sucks and Google owns them there. Their portable media player sucks and Apple owns them there. Their web browser is only alright, and there are a slew of other alternatives. Their most recent OS isn't all that great on netbooks and there are alternatives popping up there. Are you running an Microsoft web server? You'd be in a very small minority if you are. In places where Microsoft product quality isn't up to par, they pay for it.

    60. Re:Oh, this sounds like a good idea... by Runaway1956 · · Score: 1

      As in earlier posts, I get the idea that managers want a certificate stating that at some point in time, all the locks were locked, the windows were closed and latched, and that no one knows of any other means of access. And, I insist that such a certification is meaningless. Such a certification may be impressive to a potential client - IF the client doesn't have a clue.

      It doesn't matter whether it's military or not - security is indeed 24/7. For any manager to pretend otherwise is an indication that he may be otherwise incompetent.

      The result of certification can be found in the article, of course. Penetration, and loss of valuable data.

      I'm sorry, but your post, and the ones like it, are simply apologetic excuses for failure. No one should be in the business of doing a meaningless audit, just to satisfy some ridiculous requirement put in place by a clueless insurance company.

      BTW, I happen to be passingly familiar with ISO audits. Again, meaningless bullshit, meant to produce a little piece of paper and a placard whose only purpose is to impress idiots.

      --
      "Windows is like the faint smell of piss in a subway: it's there, and there's nothing you can do about it." - Charlie Br
    61. Re:Oh, this sounds like a good idea... by sumdumass · · Score: 1

      That depends on what was certified.

      If I certified your systems as PCI-DSS compliant or SOX compliant, I'm not certifying that you won't get hacked, I'm certifying that you meat a standard. Now if you were hacked because you weren't compliant and I certified you anyways, then I should have some liability. The amount of liability would be based on how much of a role the mistake I made played in the hacking.

      But if I certify you on Monday, and you replace a router on Tuesday, and you hack happened through that router on Wednesday, then I'm not liable for anything because A: I didn't certify your router or the router config and B: even if it is the same model number and make, it doesn't mean that the software is the same version, if the config is the same or that the router is not already compromised when you purchased it from Ebay at 1/3 the normal price.

      If your employee visits a porn site or some hijacked site that installs an active X control or reroutes the computers traffic through a proxy under their control and they find all the passwords and pins, it's not my problem.

      So there are a lot of details, including specific understandings, and actions or who took the actions, that need to be taken into account before external liability can be assessed.

    62. Re:Oh, this sounds like a good idea... by sumdumass · · Score: 1

      Sure, but PCI is a best practice standard, it isn't a guarantee against being hacked or any security breach. It more or less means you did the best that someone could expect at security, not that you are bullet proof.

    63. Re:Oh, this sounds like a good idea... by iamhassi · · Score: 1

      "I would argue the same should apply here."

      In other industries (e.g. the bridge building) it can also mean criminal prosecution of the auditor - for criminal negligence. I would argue the same should apply here.

      I think too many people are reading this and not understanding the responsibility of the auditor. Say I am buying a piece of gold, and I want to certify that it's 99% pure. I call some metal testing place and tell them I want it tested for purity. They say it'll cost XX amount to test it and verify it's 99% pure. I pay them, they test it and certify it. I go on and buy the gold. If I find out a week later it's not 99% pure and certification is wrong then I should be compensated in some regard.

      Same idea here. If the auditor goes through and says "I have tested all your systems and they meet PCI compliance" and I find out it does not because the auditor either missed a test or a test failed and the auditor said it passed, then I the auditor should be held liable.

      Of course the auditor should also have some kind of insurance for this, like errors and omissions insurance, which is what real estate agents, insurance agents, and many other individuals that write contracts have to cover them incase they write down or say the wrong thing. It's very cheap and very important to have.

      --
      my karma will be here long after I'm gone
    64. Re:Oh, this sounds like a good idea... by ??? · · Score: 2, Interesting

      And they failed to do that.

      They knew the processor had previously failed an audit because of storage of unencrypted PANs and non-compliant firewalls.

      They provided an audit report that said "fully compliant" with CISP.

      In the aftermath of the breach, it was discovered that the processor still had non-compliant firewalls and was still storing unencrypted PANs.

      It appears that Savvis did not do their job. This will not be the big question at the trial, though.

      Merrick was not in contractual privity with Savvis. Savvis was contracted by CardSystems, not Merrick. The issue at trial will likely be whether Savvis owed a duty of care to others that relied on their report (rather than just their client).

      I would suggest that if an audit scheme is to have any benefit at all, it must accrue to those that rely on the audit findings. If 3rd parties cannot rely on the audit findings, then there is no reason to conduct the audit in the first place.

    65. Re:Oh, this sounds like a good idea... by TemporalBeing · · Score: 1

      No, I'm not mistaking - though others may be.

      I'm also not saying that criminal prosecution should be possible for all kinds of certifications, but some certainly. And you certainly point out a difference that should be taken into account.

      If the consequences of failure by the auditor to report accurately result in (i) death to one or more people, (ii) massive financial loss to one or more entities (e.g. people, companies), (iii) failure to comply with law resulting in either of the above, etc. then yes, there should likely be criminal prosecution by the gov't alongside any lawsuits by those who relied on it.

      On the other hand, if a failure by the auditor to report accurately results in minor issues, then the lawsuits should be sufficient or at worse a fine by those who accredited the auditor.

      Now, just to be clear, when I say a "failure by the auditor to report accurately" I mean it is again a known accreditation that is current at the time of the audit as specified by the contract. So if they sign the contract saying the auditor will accredit against version 1.0, and 10 seconds later version 1.1 is release by the agency providing the audit guidelines then the auditor should be held against version 1.0, not 1.1. (That's not to say that it might be appropriate for the auditor to note it to the customer, and ensure they get audited against the right version - they should.) That is to say - there are limits.

      --
      Truth is like the sun. You can shut it out for a time, but it ain't goin' away. - Elvis Presley (source: imdb.com)
    66. Re:Oh, this sounds like a good idea... by AK+Marc · · Score: 1

      In IT, its completely different, its not tangible (well in some cases like physical security it is), but an inspector or auditor could have done his job perfectly fine, and things still went wrong as IT rules and policies, and configurations, and patches, and things like that change all the time..

      I disagree. If you "pass" a bridge, and there is a toxic waste spill that occurs that damages the structure, you would recognize that there was some act (or lack there of) that affected the certification. If you certify that policies are in place and adequate, and they are followed, and there is still a problem, you can't just say "well, stuff changes." That's the whole reason there are things like ISO 9001. If it changes, you document it. If it moves, you nail it down. If you are auditing the number of fish in a river, then holding the auditor accountable for the number of fish in it 10 years later, then yes, that's stupid. However, IT isn't random. There are policies that should be in place. There are proceedures for verifying those policies are followed. If the policies are sound and the verification is there, and everything is properly followed, then why shouldn't the auditors be responsible for their statement "this is safe"? If it is safe, then it is safe for all time with the parameters at the time of inspection. Sure, the external factors will change greatly (new threats), but the internal parameters should either not change, or only change with an approved change process to ensure there is no loss in safety.

      Unless your going to pay the auditors to run a compliance check after every change you make, they are not going to be responsible.

      You don't recertify a bridge every time a car drives over it. The auditors should outline when you should have to recertify. And some change should be possible without recertification. Stuff like painting lines on a bridge. Or changing DNS hosts. Mostly cosmetic. But if you had a throwaway password for the external server for the first DNS server and then give a core network password for the next one because it's more convenient, then you should have another audit, as you just handed out a core password to an external entity. You have to have traffic engineers looking over the bridge and they call the civil engineer if there's an issue. You can't have an audit for a point in time and then change anything you want with no order or process and hold the auditor accountable. But that just means you need a system and parameters, and if you have to keep a network engineer on staff to monitor the network and call in the auditor for changes, then that's what you should do.

      You sound more like "it's inconvenient to keep certified, so we shouldn't hold anyone to that standard." That's not how bridges work, and that's not how we should expect networks to work either.

    67. Re:Oh, this sounds like a good idea... by AK+Marc · · Score: 1

      Depends on the audit. If they weren't compliant before the audit, then the auditors told someone what to do without instruction or reason to bring them into compliance, with no documentation and no change control, then, as the auditors could reasonable expect, the network was changed back as soon as they left, then yes, the auditors are at fault. Making a company compliant so they can be rubber-stamped is not the job of auditors. That's been shown by the trouble Accenture got into for passing Enron's audit by working with the company to get the appropriate numbers displayed, rather than failing them and making them improve the process (which would have resulted in earlier discovery of Enron's shenanigans). If they aren't compliant, you can help them become compliant, but not just tweak the audit process to get a pass and leave.

      presumably Merrick can only prevail if they can show that the non-compliant that allowed the breach was also in place at the time of the audit.

      I think that if they can prove that they weren't compliant before, the auditors knew, there were tweaks done by the auditors to make them pass the audit, then the breach happened when the network was essentially in the pre-audit state, then I would say that the auditors are at fault.

    68. Re:Oh, this sounds like a good idea... by ??? · · Score: 1

      "If the banks [sic] implementation followed PCI-DSS and the auditor did its job with a generally accepted level of precision"

      It did not and they did not. Unencrypted storage of PANs, for which a previous audit had failed.

    69. Re:Oh, this sounds like a good idea... by ??? · · Score: 1

      "Unless your going to pay the auditors to run a compliance check after every change you make"

      Not relevant to the case at hand, but:

      1.1.1 A formal process for approving and testing all network connections and changes to the firewall and router configurations ...

      6.3.1 Testing of all security patches, and system and software configuration changes before deployment, including but not limited to the following:
      6.3.1.1 Validation of all input (to prevent cross-site scripting, injection flaws, malicious file execution, etc.)
      6.3.1.2 Validation of proper error handling
      6.3.1.3 Validation of secure cryptographic storage
      6.3.1.4 Validation of secure communications
      6.3.1.5 Validation of proper rolebased access control (RBAC) ...
      6.4 Follow change control
      procedures for all changes to system
      components. The procedures must
      include the following:
      6.4.1 Documentation of impact
      6.4.2 Management sign-off by appropriate parties
      6.4.3 Testing of operational functionality
      6.4.4 Back-out procedures ...
      6.6 For public-facing web applications, address new threats and vulnerabilities on an ongoing basis and ensure these applications are protected against known attacks by either of the following methods:
      - Reviewing public-facing web applications via manual or automated application vulnerability security assessment tools or methods, at least annually and after any changes
      - Installing a web-application firewall in front of public-facing web applications ...
      11.2 Run internal and external network vulnerability scans at least quarterly and after any significant change in the network (such as new system component installations, changes in network topology, firewall rule modifications, product upgrades).
      Note: Quarterly external vulnerability scans must be performed by an Approved Scanning Vendor (ASV) qualified by Payment Card Industry Security Standards Council (PCI SSC). Scans conducted after network changes may be performed by the companyâ(TM)s internal staff.
      11.3 Perform external and internal penetration testing at least once a year and after any significant infrastructure or application upgrade or modification (such as an operating system upgrade, a subnetwork added to the environment, or a web server added to the environment). These penetration tests must include the
      following:
      11.3.1 Network-layer penetration tests
      11.3.2 Application-layer penetration tests

    70. Re:Oh, this sounds like a good idea... by turbidostato · · Score: 1

      "This is a very important disctinction that some peoples fail to grasp."

      But it is one that it's very easy to spell responsibilites against.

      "An auditor basically compare a situation vs a checklist of auditable issues."

      So if he didn't check it properly, its obvious it should be held responsible for that.

      "Standards such as PCI, SOX, NERC CIPs etc. aren't designed to protect you against all known threats"

      No. But they are there to offer confidence both at the certified entity and the user of such entity, and they don't ask for peanuts to grant the certification but real money. If the certification is bollocks or the certification process as presented by the certification authority is a joke, then the certification authority should be held responsible.

      "A company can pass an audit and still be very insecure."

      If a company can pass a security audit and still be very insecure then the security certificate is a joke and the certification authority should be held liable.

      Is not that hard, really.

    71. Re:Oh, this sounds like a good idea... by turbidostato · · Score: 1

      "In reality, the auditor typically has minimal technical competency"

      His problem.

      "and is running a canned set of tools that throw out so many false positives that the reports are practically worthless"

      His problem again.

      "or if followed to the letter would make a system fail to even perform it's function."

      The problem of the certification authority and those that accepted such a certification to be valid at some realm.

      "They even may not have the a canned set of tools for the right OS in the first place, making the reports even more useless."

      The auditor's problem again.

      So we have an auditor without technical competence, without proper tools and being there to certificate over a shitty standard. But still he will ask for real money, not fake one, at the end of the transaction, won't he? Oh, how sorry I feel for that poor boy when shit hits the fan!

    72. Re:Oh, this sounds like a good idea... by turbidostato · · Score: 1

      "The big difference, its not hard to test concrete, that is one aspect of bridge building... In IT, its completely different, its not tangible"

      Well, I'd say the auditor is an adult able to measure the risk. An auditor's job is clear and their responsibilities are clear too. If money vs risk is not enough, don't work as an auditor: there's aplenty of less risky jobs over there.

      "an inspector or auditor could have done his job perfectly fine, and things still went wrong"

      That's not the auditor problem, that's not the case discussed here, and nobody on his sane mind would ask him retaliation for that.

    73. Re:Oh, this sounds like a good idea... by turbidostato · · Score: 1

      "You are essentially suggesting that audit quality does not current affect an auditor's demand"

      Probably not: conflict of interests is really obvious.

      Usually a company won't contract an auditor to check anything but to produce a certification. That's what you pay them to. Once this is understood then it follows that the auditor that makes for you easier and cheaper to get the paper is the one you want the most. If that easiness and cheapness is achieved by better audit quality, so be it; if it's achieved by other means, so be it too, and I can imagine quite a lot of ways to easy the certificate producing process totally unrelated to audit quality, and you?

    74. Re:Oh, this sounds like a good idea... by hairyfeet · · Score: 1

      I know you are probably just trying to troll, but there isn't anything wrong with Windows if you have even a tiny bit of common sense. Making my living building, selling, and repairing Windows PCs and networks I can show you plenty of customers who have been running for years without a single bug.

      The reason you get so damned many bugs on Windows is because you have so many dingbats like my customer Velma. Say hi Velma(Hi y'all!). You see Velma has a BFF Kim. And anything her BFF Kim sends her HAS to be good, as her BFF Kim is her friend and they go to Branson together once a year and anything bad must be a trick because her BFF Kim just wouldn't do that. let us watch as I interact with little Velma-

      /Me/ I don't care if that is from your BFF Kim Velma, if it is a password protected zip file it is a virus! Do NOT open that!(Velma)Oh...You worry too much! My BFF Kim wouldn't do anything like that! And see? It says happy puppy pictures! Ain't that nice? /Me/Velma as you can see that is a "happ_puppy.jpg.exe" that is a virus! Do NOT RUN THAT! (Velma) Would you calm down, drink decaf or something,it will be fine! It is from my BFF Kim! /Velma opens and runs the .exe, porn popups start flooding the screen while the network crashes from all the activity/ (Velma)Whoops. But .....It must be a trick! My BFF Kim wouldn't do something bad! /Me/.....

      So you see Zerth THIS is why Windows has so damned many bugs. It is because your friends at the Russian Business Network and their friends in Nigeria and China have figured out that it is real easy to get the Velmas to run just about anything as long as you use the right carrot. But in this case we are talking about servers, which tend to be run by somebody with a little more common sense than dear old Velma. but blaming MSFT for Velma is like blaming Winnebago for you having a wreck because you put on the cruise at 70 and went back to make a sandwich. And I'm sure if you get all the Velmas switched over to Linux your friends at the RBN will be more than capable of cooking up a "Happ_Puppy.SH" along with easy to follow instructions that Velma will happily follow if she thinks it is from her BFF Kim. It is just the dancing bunnies problem and NO OS short of a BOFH locked down thin client will help with the bunnies.

      --
      ACs don't waste your time replying, your posts are never seen by me.
    75. Re:Oh, this sounds like a good idea... by turbidostato · · Score: 1

      "As in earlier posts, I get the idea that managers want a certificate stating that at some point in time, all the locks were locked"

      Then you are wrong.

      Managers want a piece of paper that happens to increase their bussiness value. That's all.

      "I insist that such a certification is meaningless."

      That's because you started off the wrong premises. My quality is shit, I know it's shit and I want it that way. But I want my ISO-9001 certification because that means I can go after big government contracts. I got my ISO-9001 piece of paper; my quality is still shit but now I get big government contracts. Who the hell are you to say that my certification is meaningless? What would be meaningless would be to expend even a penny more than needed to achieve my goals which are to get the damn paper and nothing more.

      "security is indeed 24/7. For any manager to pretend otherwise is an indication that he may be otherwise incompetent."

      Welcome to the Real World Where Most Managers Are Indeed Incompetent (TM), marine.

    76. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 1, Interesting

      As someone can grasp from going through a pci process, or even reading much of the docs. PCI-DSS has more to do with assigning liability(away from pci members) and insurance requirements, and holding credit-card "acceptors" responsible. It might improve security, but the real focus of it seems to be to assign the responsability of fradulent online card transactions away from the card companies, by pointing it to someone else.

    77. Re:Oh, this sounds like a good idea... by Nefarious+Wheel · · Score: 1

      but blaming MSFT for Velma is like blaming Winnebago for you having a wreck because you put on the cruise at 70 and went back to make a sandwich.

      Oh dear gods what an image. I will forever think of a Happy Puppy Sandwich whenever I see a Winnebago from now on, and it's entirely your fault. Fortunately we don't see a lot of Winnebagos in Melbourne, but I wouldn't swear to the Happy Puppy Sandwiches.

      Is there a meme in here to match Natalie Portman + Grits?

      SET THE WINNEBAGO ON CRUSE, I'M HAVING A HAPPY PUPPY SANDWICH!

      Nice post, though.

      --
      Do not mock my vision of impractical footwear
    78. Re:Oh, this sounds like a good idea... by Anonymous Coward · · Score: 0

      And if the defect is something the inspector can't see because it's obscured inside the bridge structure?

      This is a well known situation in construction because so much work is hidden. Every inspectable piece of work has to be inspected before it is covered by later work. For example, inspector will come and check your foundation's rebars before you are permitted to pour the concrete.

      Nevertheless, as you mention some builders may use cheap concrete when a better one is required by the design. Then the building crumbles and the builder goes to jail.

    79. Re:Oh, this sounds like a good idea... by IBBoard · · Score: 1

      You should not just be auditing a static network at a single point in time, but also the policies and procedures for maintaing the system.

      But policies, like the parts of a car, are worn down. People push them further and further, or people outright breach them because that's the only way they can get their job done. It's all well and good saying "if you do X, Y and Z you're safe" and the company saying "we did X, Y and Z but still had a breach, so you're liable", but the chances of someone also having done W are normally somewhere between certain and guaranteed.

      As I pointed out elsewhere, certain accreditation standards are for absolute specific versions of software in a defined configuration. Any minor deviance from that and you aren't strictly guaranteed to be running at the same accreditation/assurance level.

    80. Re:Oh, this sounds like a good idea... by Zerth · · Score: 1

      Oddly enough, I wasn't talking about windows. I was talking about MSDOS, so same ballpark.

      Yes, I once worked at a place still running(under layers and layers of interfaces) on MSDOS in 200X.

      PCI asks "do you rotate your passwords". Sure we do. They're even hashed & salted. But anyone can read and write to the damn password file because it is bloody MSDOS.

      The only thing keeping that system safe is antiquity and the rational assumption nobody would be dumb enough to do such a thing.

    81. Re:Oh, this sounds like a good idea... by CHLeGrand · · Score: 1

      Actually, an auditor can audit anything you are willing to pay them to audit. If you did not pay for the auditor to examine control infrastructures at a technical level, then the problems existing there are outside the scope of the audit unless the symptoms of control weaknesses are so obvious they should be recognized in a superficial review. As yet, I have not seen auditors digging into configuration management or change controls at the level described in the ITPI's "Visible Ops" or "Visible Ops Security." Some internal auditors may address controls at that level, and there are many auditors with the technical competence to do so, but such diligence is not typical in an external audit. Audit scope may be specifically limited (as in the case of a SAS 70 review) to whatever the client wants - or does not want - the auditor to see. Audit engagements tend to be based on prevailing practice, judgement calls on risk management and compliance, and some "standard" audit techniques designed to address the more obvious controls. To date the market has not demanded or been willing to pay for audit examinations that dig deeply into security management, measurement, monitoring, and/or the cultural and political elements impacting security effectiveness. Security professionals know you can be fully compliant yet not secure. Until you give the auditors full authority (and unlimited budget) to go beyond what everyone else is doing, you will continue to get superficial point-in-time or period of time reviews with limited value. Management can, of course, commission internal audit to perform or oversee such technical reviews, but for the most part they are willing to take their chances with the status quo rather than dig deeply and find problems that may be expensive to repair. Finally there are the crooks who disable security controls to allow them to cover up money laundering, identity theft, or other crimes. In such an environment the technical depth of prevailing audit practice is not deep enough to prevent such people from covering their tracks. That is another reason auditors typically disclaim responsibility for fraud detection.

    82. Re:Oh, this sounds like a good idea... by JumpDrive · · Score: 1

      Someone has been drinking kool-aid from the Republicans.

  2. What about the Dufus? by siloko · · Score: 4, Funny

    Well much as I like people to be held responsible for the quality of their work I think it is a bit much to expect technology certification experts to be held responsible for the dufus who puts his username and password on a PostIt stuck to his monitor . . .

    1. Re:What about the Dufus? by Opportunist · · Score: 1

      Where's the problem? That dufus broke code and cert standards. If he did, that is...

      Bottom line: If there's a line in the audited security standards that reads "writing down your password is forbidden", chop the dufus' head for breaking code. If it's not, chop the auditor's for being the dufus.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    2. Re:What about the Dufus? by sorak · · Score: 1

      There may a more vague phasing involving something like "adheres to reasonable security practices", so that the auditor is covered if the user decides to yell his password out the window, or have it decorated onto a cake, or some other unpredictable and utterly retarded activity.

  3. Kind of. by Renraku · · Score: 4, Informative

    If an inspector inspects and then signs off on an elevator, and the elevator subsequently catastrophically fails due to some reason the inspector should have caught, the inspector can be held liable, unless they can show that his inspection was somehow tampered with. Like perhaps the safety interlocks were just for show and didn't have any real parts inside of them.

    Auditors should be held to the same standard, and given the same rights to defend themselves.

    I don't want to sound harsh, but considering people pay auditors to do a job, if the job isn't done right, they need to suffer the consequences.

    --
    Job? I don't have time to get a job! Who will sit around and bitch about being broke and unemployed then?
    1. Re:Kind of. by wirelessbuzzers · · Score: 5, Insightful

      I agree, but it's hard to say what standard auditors should be held to. Often, computer security audits are just surface level checks: they check your design docs and your testing methodology. And this is fine, but you get what you pay for. If a bug slips through your tests, or worse if you don't actually implement your design docs or tests, the auditors obviously shouldn't be liable. On the other hand, if there's a flaw that the auditors "should" have caught, and they don't, they should be liable at least to some degree.

      The difficulty is that full, in-depth code audits are very, very hard. Consider the Linux kernel or OpenSSL: even after 16 years of "many eyes" treatment by engineers and security researchers across the world, serious bugs keep showing up. As a result, the fact that the auditor missed something doesn't mean much, and it's not clear that a court will be able to decide whether the auditor "should" have caught it.

      I wonder if the same problem is present in other industries.

      --
      I hereby place the above post in the public domain.
    2. Re:Kind of. by Rosco+P.+Coltrane · · Score: 4, Interesting

      You're correct that, if an elevator cable is frayed and the auditor missed it, he should be sued. However, audits aren't a way for businesses to shift the blame onto the auditor: they're a way for honest businesses to confirm that everybody (employees and contractors) and everything is in order at a certain point in time. If the auditor finds something that isn't right, his job is to inform his client, and perhaps propose remedies, but that's all. It's the business' job to implement the remedies. What I mean is, audits are a tools *for the client* to help do things right, that's all.

      For instance, I once subcontracted in a company that used all manners of cracked software. A day or two before the IT audit was due, the manager used to go around telling employees to uninstall anything shady and put away copied CDs. The auditor would come, say everything was good, and the day after, all the cracked software were reinstalled. Is this the auditor's fault? The problem in this case is that the company needed the audit to be this-or-that-certified, in order to work for a certain customer. They didn't see the audit as a tool to help them do business better, but as an annoyance that could prevent them to do IT on the cheap.

      --
      "A door is what a dog is perpetually on the wrong side of" - Ogden Nash
    3. Re:Kind of. by Xest · · Score: 5, Interesting

      The problem is that auditors only check something at that point in time. They can't check that things are correct on an ongoing basis and they can't help it if what they're checking against isn't foolproof.

      I used to support IT in schools, and was sent on a PAT testing (http://www.pat-testing.info/) course so that I could PAT test equipment in schools. One thing that was made clear on the course was that if we are not willing to do PAT testing we do not have to even if our employer tells us to. Why? Because if you sign off a piece of electrical equipment as safe and someone injures themselves because it wasn't safe a day later you could be liable - that sounds fair enough at first read through, but what if it really was safe when tested but something happened after testing, before the incident that led to it becoming unsafe? How can you as an tester foresee that? I actually refused to do PAT testing because of this, I simply was not willing to sign myself as liable for something I could not control.

      Furthermore, many auditors for example, security auditors can check to ensure a company is complying to security policies, but what if those policies are flawed and a breach occurs because of that? The auditor was paid to ensure policies were followed, and it is the company that is paying for that who is at fault IMO if the policy wasn't enough. Say an IT security policy states that all security patches should be applied immediately, that's great, a security auditor could check that, but what if then there's a breach using a vulnerability for which there was no patch? Is it the auditors fault?

      To me it's the company's fault again, the real problem is this, companies don't want to spend time and money on things they see no instant benefit from such as following security policies and procedures. They do the bare minimum they can and comply with the policies and procedures they have to - knowing full well that these policies and procedures are the bare minimum and insufficient for real security and good practice. There's always more that can be done, allowing them to shift the blame just means they'll struggle to find auditors.

      Auditors do what auditors are supposed to do, if auditors do their job wrong then sure they should be liable, but I do not see how you can make them liable for something outside their remit. If you pay someone for a full security audit it's one thing, if however you pay them to ensure you're BS7799 compliant and you don't do anything over and above that but suffer a breach as a result of the fact there are things you can do over and above BS7799 then it's your companies fault.

      The answer has to come down to the auditor's role, and if the auditor has audited what he's supposed to he should not be at fault. It is only when the auditor has accepted to do an audit and signed it off and that his audit was found to be at fault that he should be liable. In the example of the lift you state though, there is no way that we can know if the auditor was at fault, if he tested it and it really was safe, how could he be at fault if say over night a minor earthquake occured making the lift not safe? What if because of the nature of it he can't prove that it wasn't like that when he tested it? Should he be jailed for manslaughter? When he did nothing wrong at all, should he even have to suffer having his name dragged through the mud, possibly being suspended from work/losing his job in the process until he's finally found not guilty even though his life is wrecked anyway?

      Companies should be held liable anyway, if a company gets screwed by a bad auditor it should be on the company to prove the audit itself was faulty. In other words, let's stick to innocent until proven guilty. If a company feels the auditor is guilty, let them prove it, not vice versa.

    4. Re:Kind of. by Renraku · · Score: 1

      Correct. Audits and inspections are always point-in-time snapshots of the state of whatever is being audited or inspected. It should be held accordingly. Auditors and inspectors can not be held accountable for things found out after the inspection. Like that steel really does have shitty 60-year durability specs or that bind is a buggy piece of shit.

      All of that being said, there's no reason an inspector should sign off on a system with open shares and no firewall or a bridge with eroded foundations.

      --
      Job? I don't have time to get a job! Who will sit around and bitch about being broke and unemployed then?
    5. Re:Kind of. by olman · · Score: 1

      You're correct that, if an elevator cable is frayed and the auditor missed it, he should be sued. However, audits aren't a way for businesses to shift the blame onto the auditor: they're a way for honest businesses to confirm that everybody (employees and contractors) and everything is in order at a certain point in time. If the auditor finds something that isn't right, his job is to inform his client, and perhaps propose remedies, but that's all. It's the business' job to implement the remedies. What I mean is, audits are a tools *for the client* to help do things right, that's all.

      Nb. Following pertains to european regulations. Yes, it's the continent not on same map page as US.

      On the subject of auditing machines and devices, now the demand is suddenly the auditor/inspector has personally checked every single critical component making up the elevator? And their installation?

      Right. Maybe Superman with his X-Ray vision could give decent go-ahead on an installation with superficial examination and checking the paperwork. Ordinary people obviously can't.

      However much you'd love to shift the blame, it's still the responsibility of the manufacturer and installer of the elevator that the elevator is safe to use. Auditors can only verify the company manufactoring practices and QA are according to the required standards. On paper.

      What this auditing and certifying actually does is alleviate the blame on the people doing the installation. If there's an accident, you can be sure the setup will be examined thoroughly to find out what exactly went wrong and why. But someone can be actually sued if it can be demonstrated there was criminal negligence or fraudulent practices.

      In any case systems like elevators have huge safety margins and strict regulation. Whereas contruction industry as a whole is known for integrity and following standards to the letter (cough) consequences for using substandard equipment or installations where people are liable to be hurt or killed are fairly severe.

      After all strict quality management practices and certifications exist so that manufacturers and builders are not able to endanger lives as a cost cutting measure.

      Accidents do happen and then there is a chain of blame and it's not not usually the auditor or inspector ending up holding the bag. Supervisors are far more likely to get the blame who actually oversee the construction.

    6. Re:Kind of. by Xest · · Score: 1

      "All of that being said, there's no reason an inspector should sign off on a system with open shares and no firewall or a bridge with eroded foundations."

      Well, as I say, I guess that depends if he's being paid to do a general security audit, or if he's being paid to ensure the company adheres to a specific standard, and the standard in question doesn't specify that he should check that. I think as I say, the issue is, a lot of standards are quite weak, although maybe not to that extreme, some companies seem to think they can adhere to standard X and then avoid any responsibility without actually evaluating how useful standard X really is in achieving the goal.

      It's a little like in the UK, the whole MP expenses scandal where we keep being told by the MPs "but it was within the rules". That doesn't cut it with me, and a company losing my personal details to criminals saying "yeah but we followed standard X" doesn't cut it with me if standard X wasn't truly sufficient to protect my details and more could reasonably have been done.

    7. Re:Kind of. by Tom · · Score: 4, Informative

      The problem is that auditors only check something at that point in time. They can't check that things are correct on an ongoing basis and they can't help it if what they're checking against isn't foolproof.

      The elevator guy has the same problem and yet it works in real life.

      That is because in any real life situation, tests are indeed done repeatedly, such as every quarter, every month - or if they are really important, every day or every event. No plane in the western world takes off without the pilot and co-pilot having run through a standardized checklist first.

      "But things can change" is a pretty bad excuse. Like the elevator (where wear and tear change the physics constantly), your system has to be resilient enough to withstand normal changes (e.g. wear and tear, different weights, etc.) at least until the next check. Unauthorized changes have to be hard to make unintentionally (that's why there's no "cut the cable" button inside the elevator).

      It really isn't that hard. It works in thousands of areas, many of whom are non-trivial and technically complex (e.g. airplanes). But for some reason, we think it's impossible to do it in auditing and software?

      --
      Assorted stuff I do sometimes: Lemuria.org
    8. Re:Kind of. by Aladrin · · Score: 1

      It occurs to me that the loss of productivity for those few days around every audit is probably costly enough to just pay for all the stolen software. This is a really, really bonehead move.

      --
      "If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
    9. Re:Kind of. by Xest · · Score: 1

      I think you're largely missing my point in the context of your responses.

      In real life, the situation with elevators and planes work precisely because the companies are held responsible when something goes wrong as I stated.

      But this isn't what's being suggested. What's being suggested is that a company should be able to pay for infrequent checks on something that needs to be checked frequently and blame the person who performs those infrequent checks because the infrequent checking led to an unacceptable outcome.

      I advocate the model used in the airline industry, where things that are critical and important should be checked regularly and that companies should not be able to cut costs by avoiding infrequent checks whilst avoiding responsibility for this.

      Again, airlines pay people to check frequently enough to avoid issues, this is not the same as what the bank in TFA is doing - they paid them once to certify them as following a set of standard procedures, they didn't pay them to come in and ensure they're following them every single day.

      You're right, it can work in software and auditing, but not on the cheap as most companies are trying to get away with. It's going to cost just like it does in the airline business to have their own set of technicians doing the checks round the clock, they have to accept that.

    10. Re:Kind of. by Opportunist · · Score: 2, Interesting

      Oh tell me about it.

      I've done more than one security test for companies that boast 27001 certs, only to succeed with the most basic systems of social engineering or inside jobs. More often than not I get paid to shut my mouth rather than talk about how stellar the security of the company is.

      The general reactions are quite different, too. Some companies are genuinely interested in security and they're quite happy when you find a loophole in their process. Most, though, just want a signed paper and get rid of it.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    11. Re:Kind of. by Opportunist · · Score: 1

      My experience is pretty much the same, audits are done when a company need a certain cert to get a (often public) contract or to comply with legal requirements. Nothing else. No company I had been involved with in audits did it "on their own", because they wanted to better themselves. Audits and the certs that come with them are usually seen as a necessary evil, not something desireable.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    12. Re:Kind of. by Opportunist · · Score: 1

      That doesn't cut it with me, and a company losing my personal details to criminals saying "yeah but we followed standard X" doesn't cut it with me if standard X wasn't truly sufficient to protect my details and more could reasonably have been done.

      Don't blame the company, blame the government for not requiring a better standard.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    13. Re:Kind of. by Opportunist · · Score: 1

      "Things can change" is actually a pretty good excuse when it comes to IT security. Because they do change, and quickly they do. Here it's reverse, it's usually not the system that wears and tears, it's the standard that becomes a joke when pitted against ever changing and evolving threats.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    14. Re:Kind of. by Xest · · Score: 1

      But isn't that just the same problem I mention? The idea you can get away with something as long as it's in the rules even if the results are extremely damaging as per the MP expenses scandal in the UK?

      Should we really have to expect the government to legislate everything? Isn't it better that companies are running scared over this sort of thing, such that when they can't be bothered to do the important things properly - like security and something goes wrong, that they're held liable?

      I fear if we let companies get away with things just because the government hasn't legislated on them we're going to end up in a world of shit.

      Both individuals and companies should be kept responsible for effects of their actions or inaction.

    15. Re:Kind of. by Opportunist · · Score: 1

      I'd just fear that the fallout of a "company is liable for everything" formula would be that small companies would have to bear the brunt of it while large corps managed to weasel out of it.

      A company has to store certain personal data of customers, it's a basic necessity of business. You have a list of customers, you have your supplyers, you have to store what they owe you and what they paid. Most of it is even required by law, so you can be taxed accordingly. So the mantra "don't store data and you're safe" won't work out. To do that, you have to shut down your company.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    16. Re:Kind of. by Anonymous Coward · · Score: 0

      It's not that it's impossible to do in auditing and software at all... it's entirely feasible. The issue is cost - those checks (by pilots, co-pilots, et al) and required maintenance cost airlines a significant amount of money, but it is paid because: regulations say they have to, and the cost of a rash of failures is terrible PR. In the IT world, companies want to be certified, only because they have to, and don't want to spend much money on that compliance. That's totally ignoring the companies (typically small to medium sized) that don't have the resources to expend on "doing the audit right". It's apples and oranges.

    17. Re:Kind of. by Xest · · Score: 1

      I'm not convinced they do. Certainly there's absolutely no reason they need to store your credit card details for example unless you explicitly ask them to for your convenience, yet some do store them.

      There's also little reason stuff that needs to be stored for archive reasons can't be shipped off to an offline system. At very least, a backend providing data to the front end should limit the amount of data that can be pulled so that if suddenly the web server requests massive volumes of customer data (i.e. an obvious hack) it refuses. It doesn't need to be stored on an internet accessible system and it certainly doesn't need to be stored unencrypted.

      None of these solutions are cost prohibitive for small companies and yet time and time again companies manage to get this data pulled from their servers en-masse.

      There's still a lot of data that doesn't need to be stored out there that companies aren't held liable for when it goes missing, and the data that is stored is still often stored in an unacceptably insecure manner.

    18. Re:Kind of. by Tom · · Score: 2, Insightful

      It's not that it's impossible to do in auditing and software at all... it's entirely feasible. The issue is cost - those checks (by pilots, co-pilots, et al) and required maintenance cost airlines a significant amount of money, but it is paid because: regulations say they have to, and the cost of a rash of failures is terrible PR. In the IT world, companies want to be certified, only because they have to, and don't want to spend much money on that compliance. That's totally ignoring the companies (typically small to medium sized) that don't have the resources to expend on "doing the audit right". It's apples and oranges.

      No, it doesn't.

      If you are "too small" and "don't have the resources" to fly a plane safely, then you can't play in the commercial airlines market. Tough luck, but we're all better off this way, thank you.

      Now there are other markets, where quality isn't that important, and failure not half as critical. You could become a hairdresser, for example. Could still ruin someone's looks, but not their life. That's why hairdressers do not have to check all their equipment before cutting your hair.

      And I dare to claim that software in many appliances is quite capable of ruining not one life, and not hundreds, but many thousands, or whole populations. Think of the software that drives the stock markets. Same for auditing. If you audited a bank in 2008 and you didn't notice that it's all a huge house of cards that's going to come tumbling down sometime in the near future, then you didn't do your job properly.

      --
      Assorted stuff I do sometimes: Lemuria.org
    19. Re:Kind of. by Tom · · Score: 1

      "Things can change" is actually a pretty good excuse when it comes to IT security. Because they do change, and quickly they do.

      That's vastly overestimated. The majority of security problems are not the 0day exploits. The majority of security problems are old bugs, outdated software, bad procedures and users not trained in or unaware of security issues.

      --
      Assorted stuff I do sometimes: Lemuria.org
    20. Re:Kind of. by Tom · · Score: 1

      I advocate the model used in the airline industry, where things that are critical and important should be checked regularly and that companies should not be able to cut costs by avoiding infrequent checks whilst avoiding responsibility for this.

      Ok, I'm with you on that.

      You're right, it can work in software and auditing, but not on the cheap as most companies are trying to get away with. It's going to cost just like it does in the airline business to have their own set of technicians doing the checks round the clock, they have to accept that.

      Absolutely. It doesn't even have to be horribly expensive. One less trip to a nice cozy beachside "meeting" for upper management would probably save enough to pay for the entire thing.

      --
      Assorted stuff I do sometimes: Lemuria.org
    21. Re:Kind of. by Opportunist · · Score: 1

      By and large true, what I meant is something different. Most security certs don't take into account things like buffer overflows and generally malformed user data. For example, you usually don't see any procedure for things like how to handle data files because they were not considered a threat when the certification standards were assembled.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    22. Re:Kind of. by Anonymous Coward · · Score: 0

      Sure, so the 9/11 pilots were negligent, right? Or the ground crew? Because after their checks, the planes should have been safe until the next checks.

    23. Re:Kind of. by Tom · · Score: 1

      Sure, so the 9/11 pilots were negligent, right? Or the ground crew? Because after their checks, the planes should have been safe until the next checks.

      Which they were, troll.

      The fact that I can smash my iPhone with a brick doesn't mean there's a fault in its software.

      (for the non-trolling audience: The planes worked perfectly as expected. The passengers didn't, but they're not within the pilots area of responsibility.)

      --
      Assorted stuff I do sometimes: Lemuria.org
    24. Re:Kind of. by Tom · · Score: 1

      That's correct. My SOX controls do not contain an objective that says "software must be bug-free".

      However, they do contain controls that ensure that the tech people are up to date information-wise, and the systems are up to date patch-wise. Which is the next best thing.

      Likewise, data handling is far below the scope of most audits. This is the area of software testing, pentesting, QA, etc. - the audit process's part in the game is to ensure that these things are done.

      --
      Assorted stuff I do sometimes: Lemuria.org
    25. Re:Kind of. by Anonymous Coward · · Score: 0

      As someone who works for a PCI certified compliant company, I'm of the opinion that the certification is rather inherently flawed. First off, anything that is considered "outside the scope of the audit" isn't even investigated, even if it's on the same network; and the auditors don't make that decision, the company does (the auditors are usually not even told about it). So as long as the auditors don't see it, a botnet controlled PC can live on your network and not prevent certification. Second, most of the time the audits are Windows-centric, and when the auditors ask questions about non-Windows servers that don't have answers they often fall "off the rails" as it were and don't know how to handle it. This even happens with unanticipated things on the Windows systens! And third, systems don't have to be secure, they just have to not have holes in security-related pieces. As an example, SMTP mail transport, which must be assumed to be insecure with its very nature of store-and-forward, cannot allow SSLv2 ciphers if it does STARTTLS as the ciphers can be broken. However, it is perfectly acceptable to not do ANY form of TLS encryption at the SMTP transport level, because there are no security holes to exploit. And while I can see the argument that if you're going to allow encryption you shouldn't use a vulnerable version, preferring no encryption over the possibility of weak encryption on a known insecure connection (which other parts of the standard disallow unencrypted message data to be transported over) seems pretty dumb to me...

      And lets not even mention how much stuff that we're supposed to be doing regularly but don't necessarily have time for (because we have to do other things to keep the business running) get bumped up to high priority in the weeks leading up to an audit. Audits aren't necessarily a bad thing (if nothing else, sometimes they point out things you've missed and you then fix them), but rarely will failing a piece of an audit prevent you from passing as a whole - they just tell you to "remediate" it, and check it off when you tell them you did it (usually requiring some sort of evidence, but evidence can be "massaged" or even fabricated outright). Plus, a lot of audit interviews are just like lawsuit depositions - you answer the question precisely, and don't tell them _anything_ else that might get them to ask more questions - so the value of an audit from a security standpoint is entirely dependent on the intent of the entity being audited. If it were truly to enhance security, it might just do that; but if the whole purpose is to get the auditors to go away as quickly as possible and give you a passing grade, they're all but worthless in the long run...

  4. I think so by Hecatonchires · · Score: 1

    But there'll be an indemnity or escape clause in their contract with the processor.

    --

    Yay me!

  5. Costs... by Bert64 · · Score: 3, Insightful

    All it will do, is make future certifications 10 times slower, more invasive and more expensive... This bank is shooting themselves in the foot because they will have to get themselves certified again in the future and will be expected to pay a hefty premium.

    Besides, the auditor merely certifies that a particular defined system complies with a given spec at a point in time... They don't assert that the setup is secure, merely that it complies with the letter of the standard, and most of these standards are poorly written with loopholes big enough to drive a truck through.

    Not to mention that there are ongoing changes, such as patching and updates to signature files etc, do you need to recertify every time a minor change is made? A minor change could introduce vulnerabilities, for instance a security update could introduce new features and bring with it new exploitable issues while it also fixes an older issue.

    How widely do you define the scope? ideally you would include absolutely everything associated with the system, so every workstation used for admin purposes, every inch of cabling etc, this would make the scope very large and costly to deal with.

    And how about the age old question of human error? No matter how secure a system is, an error (or intentional attack) by the legitimate users could break things in all manner of ways.

    --
    http://spamdecoy.net - free throwaway anonymous email - avoid spam!
    1. Re:Costs... by IBBoard · · Score: 1

      Not to mention that there are ongoing changes, such as patching and updates to signature files etc, do you need to recertify every time a minor change is made? A minor change could introduce vulnerabilities, for instance a security update could introduce new features and bring with it new exploitable issues while it also fixes an older issue.

      That's a problem with people relying on EALs to assure their hardware and software in high security environments. If Oracle 9 got EAL5 accreditation, you can't just take any version of Oracle 9, slap it in your EAL5 system and be sure that the database is EAL5 approved. There are a number of other factors that make up the EAL (and probably other) review processes, and they tend to be specific to the exact build and the configuration of the app and OS at the time. If you don't have the same build with the same configuration then you could have vulnerabilities and not get the same EAL.

    2. Re:Costs... by Anonymous Coward · · Score: 0

      No it won't blow out costs. It'd just stop auditors from not doing the job they're supposed to be doing.

      Firstly to answer your question about minor changes: No you don't have to recertify for patches, but you need to have a patch management policy in place at time of certification. Secondly, the scope is quite clear - everything in the same network segment as the cardholder data. That is in the introduction of the PCI DSS - you can read it at pcisecuritystandards.org.

      Thirdly, that is not what (as far as I can tell) this lawsuit is about. The auditors said the company was compliant but the auditors were lying. The company wasn't compliant. They never were.

      Seriously, this kind of shit is rife in the auditing industry. I know the company I work for has come in to do reassessments for PCI DSS and we quite often find that the previous auditors simply signed them off without them being close to compliant. The original auditors weren't small dodgy companies either, they were huge multinationals.

      In these cases this is not corruption on the part of the certified company. They honestly believed that they were compliant because they were told that they were. They didn't know that if they had a breach then they would have been hung up to dry and the fault would be with the auditors.

      So tell me, why shouldn't the auditors be liable for what is, in effect, lying?

    3. Re:Costs... by Twillerror · · Score: 1

      Will any certification companies even work with them now? It is the right of any business to refuse service to any other business correct? If they cannot find a company to certify them in PCI then they will loose their rights to process cards right?

    4. Re:Costs... by Bert64 · · Score: 1

      Absolutely, if the auditors can be proven to be lying or incompetent then they should be held liable.

      I was just pointing out that the standards themselves are often flawed, and it's quite possible to be compliant while still being insecure.. As you pointed out, a patching policy is part of the PCI DSS standard, but it doesn't take into account patches which introduce new functionality. Personally i do think that any patches which do anything more than directly fix known vulnerabilities should be re-certified.
      Also "the same network segment as the cardholder data", but what network segments which have access to the segment holding the card data? All it takes is for one admin workstation to be compromised...

      As you point out tho, there are many incompetent companies certifying for PCI among other things... They get away with it because generally nothing bad happens, or isn't discovered, and these companies which cut corners are cheaper... It's also costly to actually do the work required to comply with the standard, so having an incompetent auditor saves you money.

      --
      http://spamdecoy.net - free throwaway anonymous email - avoid spam!
  6. Liable for what, exactly? by getuid() · · Score: 5, Insightful

    Should the auditor be liable for mis-certification? Or for the (correctly) certified system not withstanding attacks?

    I think people should *very* hard try to distinguish between the two scenarios:

    1) An auditor certifies a system as XY-compliant as of [insert date here]. However, it can be demonstrated that the system was *not* XY-compliant at that date.

    2) An auditor certifies a system as XY-compliant as of [insert date here]. However, at a later date, the system breaks for some reason. It can be proven that the system was XY-compliant, but for some reason (stupid user interaction?) is not anymore. Or, even better: it can be proven that the system *still* is XY-compliant, but the XY-standard is unfit to defend [insert attack here].

    I think in case (1) the auditor should be held liable, since he obviously certified something that didn't meet the promised standards. However, in case of (2), not the auditor is to blame. If the system breaks despite of the certification, then it's not the auditor's fault -- it's how things work, and making a scapegoat out of the auditor is not going to do anybody any good. Even worse, if the system fails to meet standard XY because a stupid user (or admin, for that matter) interaction *after* the certification, then there's no way an auditor could have prevented that -- it's either the user/admin's fault for interfering with a certified system, or the standard's fault for not defining what a user/admin is allowed to do with the system without interfering with its certified qualities.

    1. Re:Liable for what, exactly? by Tom · · Score: 2, Insightful

      I think people should *very* hard try to distinguish between the two scenarios:

      I think people should try harder to understand auditing.

      Static audits are a thing of the past. Every audit and compliance proceduree AD 2009 includes not only checks of the current system state, but in fact puts more of a focus on changes. More precisely: Change management. In a properly certified and audited system, it ought to be impossible to change the system in a compliant way into a non-compliant state. Either your changes are part of the proper change procedures, or they are not. If they are not, then you (i.e. the guy doing the non-compliant change) is responsible, because you broke procedure. If the change is OK, but its effects are not, then the change process is faulty and whoever audited it didn't catch that.

      One way or the other, you very clearly find a culprit if you include change management into your processes.

      And any auditing that (2009) gets signed off without containing change management should never have been signed off in the first place, so again the auditor is clearly at fault.

      --
      Assorted stuff I do sometimes: Lemuria.org
    2. Re:Liable for what, exactly? by herske · · Score: 1

      From my experience, auditing is always a trade-off between standards and the client's particular conditions. The audit are as demanding as the customer's budget. I am sure that the auditors can set the bar so high that only few customers would get certification. They don't do it and they won't do it, because there is demand for "good enough" certification at reasonable fees and this is what the auditors deliver.

    3. Re:Liable for what, exactly? by getuid() · · Score: 1

      And any auditing that (2009) gets signed off without containing change management should never have been signed off in the first place, so again the auditor is clearly at fault.

      (I'm asking out of curiosity, not to troll you :-)

      Maybe I'm mistaken, but isn't *any* auditing a check of the state? Even a check of a process (for example an audit checking the change strategy) in fact checks the *state* of the rules to be followed when applying a change. Doesn't it?

      Now: what's the job of an auditor? Is he (a) to certify that a certain system/proces/whatever meets a given standard, or (b) is he to certify that a system/proces/whatever *is* something? (Think: is "unbreakable"...).

      I always thought of an "auditor" as of someone who does (a) -- for security reasons, for quality management, etc...

      Now, if a given system meets a given standard and is *certified* as meeting that standard, then the auditor is out of trouble -- isn't he? If the standard is good, then "compliance" is probably pretty well defined and should be (relatively) unamiguously clear whether a system is or is not compliant. In this case, the auditor never laid his head on the line for the *quality* of a given system, or it's fitness for a particular purpose. The auditor "only" signs for compliance with a particular standard, and, if he does his job well, everything else is the standard's or the user's fault.

      However, if an "auditor" has to do (b), then... he's basically fsck'ed. Same as (a) for a poorly defined standard. ...or where's my thinking error?

    4. Re:Liable for what, exactly? by Tom · · Score: 1

      Maybe I'm mistaken, but isn't *any* auditing a check of the state? Even a check of a process (for example an audit checking the change strategy) in fact checks the *state* of the rules to be followed when applying a change. Doesn't it?

      Yes, and no.

      What usually happens is that you define your target state, and check if the current state matches. If not, you add remediation plans to make it match or reach the objective in different ways.

      So while you do check your process at time X, the actual control says something like "A change management process tracks, documents and authorizes all changes to the system." while a second control says something like "The system is checked every X days/weeks/months to verify that no unauthorized changes have occured."

      So while the actual audit is on date X, it usually does check the history. So your auditor will come in, and in addition to checking the system state today, he might as for your changelogs from January and April. As well as your documentation of testing from, say, February.

      Now: what's the job of an auditor? Is he (a) to certify that a certain system/proces/whatever meets a given standard, or (b) is he to certify that a system/proces/whatever *is* something? (Think: is "unbreakable"...).

      Depends on who you ask :-)

      Usually, the job of the auditor is to verify the effectiveness of your controls. In other words, he's the guy who "watches the watchers". Remember that you as a company are the one who should notice any anomalities in the first place. What the auditor does is verify that you have an effective infrastructure, processes, etc. in place to be able to do that.

      Which is why when Joe Dumb smuggles a change past the change management and puts it online, then Joe is the culprit. But if there's no change management process in the first place, then management is the culprit. And if there is a change management process, but it's all bullocks and doesn't do a thing, that is what the auditor should've noticed, and what he should be held responsible for if not.

      --
      Assorted stuff I do sometimes: Lemuria.org
    5. Re:Liable for what, exactly? by jazzkat · · Score: 1

      Excellent differential, getuid(). If the bank is suing for #2, then I have a question: how can any system based on Microsoft Windows - or any other Microsoft product - be certified as DSS/PCI compliant, ever? There is too much under Windows that Microsoft still controls. The auditor would have to recertify with every Microsoft update. This would cause further problems as updates were delayed due to auditing, or because the merchant could not afford to audit.

    6. Re:Liable for what, exactly? by Anonymous Coward · · Score: 0

      Even in your case (1) the liability isn't certain. If an auditor performs random sampling of a population to gain reasonable assurance that some control or standard is met, the auditor may still miss the small portion of cases where the system fails. Hence in audit reports most language borders on ambiguous since an auditor can never provide complete assurance.

    7. Re:Liable for what, exactly? by mahadiga · · Score: 1

      Isn't this logic applicable everywhere? What is the solution if Mr. President (Obama) fails to deliver in next 4 years?

      --
      I'd like to buy homeland for our 10 million people. http://twitter.com/mahadiga
  7. Just problems by MortenMW · · Score: 1

    Who will ever even attempt to certify this bank again? If the auditor made a mistake I can understand the bank, but if the problem was caused by a user, I can not see how to auditor is responsible...

  8. Wow by Peregr1n · · Score: 4, Insightful

    The big banks really are intent on shooting themselves in the foot. If they hold the auditor liable for security breaches, nobody else will be willing to offer certification services for PCI-DSS. And considering that it's the banks who desperately want everyone to be PCI-DSS compliant (does anybody other than the banks get any benefit from it? Really?), that is particularly stupid.
    It's hard enough achieving compliancy as it is - whenever we get near to completing the questionnaire, they change all the questions!

    1. Re:Wow by Anonymous Coward · · Score: 0

      What about banks funding R&D on systems security?

    2. Re:Wow by mokeyboy · · Score: 1

      If an auditor certifies a system compliant, at a set point in time, to an agreed, contractually stated structure of compliance, how is this different from an insurance agency underwriting the contract to a set event misfortune? If there is no effective penalty mechanism, does this not just encourage the types of behavior most recently lambasted during the GFC? Operators in this sphere are well rewarded for their efforts. Why should they not stand by their assessments (ie fiscal risk) in addition to reputation loss? Any other contract scenario would require it so why not this circumstance?

    3. Re:Wow by umghhh · · Score: 1

      Maybe the security really should cost more. I do not mean throwing money at the problem in hope it goes away but any reasonable security system will costs money - for audits and for implementation of findings etc. Maybe exactly that was a problem - TFA does not say exactly why they sue - maybe they know the auditors failed to inform them about something or they just want to fork the costs of the security breach and its repair.

    4. Re:Wow by columbus · · Score: 1

      (does anybody other than the banks get any benefit from it? Really?)

      Yes. The credit card companies benefit. Visa & Mastercard, not citigroup or another issuing bank that puts the visa logo on a credit line held by the bank. PCI DSS is a big proactive shield that the credit card companies can hold up before congress and say:

      "See? We are increasing the level of security. We are a self regulating industry. There is no need for you to get involved & legislate security and in fact it would be hurtful because one size does not fit all".

      At the same time, the credit card companies move all of the cost of PCI DSS to merchants, service providers & banks. They get to take the lion's share of the money and do none of the work.

      Of course I could be wrong and all of this is just my opinion. But it is an informed opinion; I was the chief security officer of a small company working through the process of becoming PCI compliant as a level 1 service provider (a designation that the company received).

      But for congress to not get involved, PCI has to be perceived to be effective. (PCI compliance is not the same as security; one can be compliant and still be insecure). So whenever a 'compliant' company gets hacked, the blame game starts. The credit card companies, the security assesors, and the third party scanning providers all need to maintain the illusion that PCI is effective - that a truly PCI compliant company is invulnerable. So they shift the blame back to the victim (in this case, the bank). 'Our post-breach forensic analysis revealed that you were not truly PCI compliant even though we said you were earlier'.

      The trick is interpretation. Anyone who has attempted to implement the PCI DSS requirements knows that the devil is in the details and that there is a lot of grey area when a theoretical standard meets practical implementation. The questions arising from that grey area are resolved by an interpretation by the auditor of the requirements. But the interpretation can be fickle.

      Need to change the compliance status of a compliant company post-breach? Simple. Change the interpretation. A compliant company is suddenly non-compliant with no physical changes whatsoever. PCI gets to maintain its 100% success rate.

      It's illusion. 3 parts out of 5 of the whole PCI compliance process is security theatre.

      The fundamental problem is that payment by credit card is insecure. Always has been. Always will be.

      Why? Because of the tradeoff between security and convenience. The credit card companies are in the business of selling convenience & it is not in their interest to sell security instead because there is not much of a market for it (although there is plenty of a market for the illusion of security). I don't believe that they will adopt a secure payment method until legislation forces their hand. If market forces were going to do it, they would have done it already. Hundreds of millions of cardholder accounts have been compromised within the space of a few years and the ecosystem is still insecure, nor is it looking to become more secure any time soon.
       

      --
      friends don't let friends teleport drunk
  9. Due dilligence. by TapeCutter · · Score: 1

    "If they win this lawsuit, they're setting a dangerous precedent"

    Audits are performed so the company can demonstrate due dilligence should something go wrong, if the auditors themselves cannot show due dilligence in their own actions then they deserve to be hammered.

    --
    And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
    1. Re:Due dilligence. by mangobrain · · Score: 1

      Yes, but to know whether or not the auditors were sufficiently diligent, you have to define precisely what their role is.

      Is their role to certify that at the time of testing, the company being tested met the required standards?

      Or, is their role to certify that the company met the required standards at time of testing, and continues to meet them for as long as the certification stands?

      The two are significantly different. In reality, what they actually do is probably the former, and hence they probably only have the necessary records to demonstrate that they did their job for one point in time. The interesting thing here is that, seemingly, it is not yet obvious from a legal perspective which of the two the certifiers *should* be doing. If the court decides that they *should* have been doing the latter, but only did the former, then all hell breaks loose.

    2. Re:Due dilligence. by mysidia · · Score: 1

      Also: What happens if they met the standards and were compromised anyways, because mere compliance with the standards were inadequate to make their systems secure?

    3. Re:Due dilligence. by TapeCutter · · Score: 1

      "Is their role to certify that at the time of testing, the company being tested met the required standards? Or, is their role to certify that the company met the required standards at time of testing, and continues to meet them for as long as the certification stands?"

      It can be either, it depends on what service they were hired to provide. Normally a cert is a snapshot that shows the company met the standard when they were audited and for it to remain relevant the company need to be certified at regular intervals. Just like my BSc shows that I understood partial differentials at the time I was tested (20yrs ago) but because I'm not tested regularly it cannot be assumed I still understand it. A cert is not about being right it's about doing whatever it is your doing in an accountable manner.

      --
      And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
    4. Re:Due dilligence. by TapeCutter · · Score: 1

      Same thing that happens to an airline who complied with all matainence standards and their plane still dropped out of the sky, nothing. However on such occasions the authours of the standard must fix it so as to avoid a repeat. In the airline industry they go to great lengths and expense to do so but planes will still fall out the sky simply because life is a shit sandwich and we all have to take a bite.

      In other words; standards attempt to prevent known problems (due dilligence), it's illogical to expect them to do anything to prevent unknown problems (fortune telling). The law reflects that logic.

      --
      And did you exchange a walk on part in the war for a lead role in a cage? - Pink Floyd.
    5. Re:Due dilligence. by DrLang21 · · Score: 1

      An auditor should always demand requirements documentation from the customer before certifying anything. If there are no requirements, then what are you certifying against? If an auditor has requirements, and blatantly failed to verify or validate all of them, that's negligence.

      --
      I see the glass as full with a FoS of 2.
    6. Re:Due dilligence. by AK+Marc · · Score: 1

      That's a great example. If a plane is inspected and found sound, then it's certified. If there's maintenance (not done by the same level person as the full inspections) and they don't follow the proceedure, then it's no longer certified. If there's a maintenance process and it's followed, it remains certified, even after changes. If they have a plane that passed its last full inspection, but had improper maintenance on it, they will have their asses handed to them. But in that case, there is a change process, a maintenance process, and that's set up essentially by the group that licenses the audtors. So whatever audit they were being audited for should be an ongoing audit that is valid into the future, with restrictions and such. That's how they work for everything except finances...

  10. I am on neither side by daeglin · · Score: 1

    From what I have heard and seen, auditors do a very lousy job. I very much hate the fact that they get a lot money while they are generally not responsible for the quality of their work.

    On the other hand, it is clear that auditors can not find all possible problems, therefore it doesn't make sense to make them responsible for all incidents. This just would not work.

  11. Not a problem by Anonymous Coward · · Score: 1, Funny

    IANAL, but as far as I recall there is the SCOTUS decision in Smoremberg vs. Entertaining Dance Clothing Corp. where the widow of a man sued a textile cooperation because her husband accidentially strangulated himself with the power cord of a power drill by slipping from a ladder while repairing the roof of his garage and wearing pink ballet shoes and a pink tutu.
    The layers of the widow argued that the shoes were certified as "safe", but the company argued that this only referred to normal ballet dancing and not home repairs with power tools.

    The very same argument could be applied here.

    1. Re:Not a problem by Anonymous Coward · · Score: 0

      The layers of the widow argued that the shoes were certified as "safe"

      Eww, TMI.

    2. Re:Not a problem by Anonymous Coward · · Score: 0

      IANAL, but as far as I recall there is the SCOTUS decision in Smoremberg vs. Entertaining Dance Clothing Corp

      Hey, AC, howzabout a cite for that case, I can't find it.

  12. Are standards deterministic? by HetMes · · Score: 1

    Are there certain parts of a standard that leave enough room for interpretation by the auditor to warrant an lawsuit if in hindsight this interpretation may be demonstrably flawed? Or, given all relevant information, will different auditors, following their auditing standards (damn, circular), reach the same conclusion?

  13. Auditing by mork · · Score: 1

    Auditors check that the company have security policies, that they have proper procedures and that these have been followed in the past. There is obviously no guarantee that the employees in the company will continue to follow the security procedures, just because they have done so in the past. Security breaches usually occur because someone failed to follow procedure.

    Security standards and audits give the company assurance that they have reduced the chance of security breaches as much as possible. However , you can NEVER certify any system as "secure".
    Audits usually control access and change procedures for systems and verify that there are controls and procedures that have been followed up to that point in time.

    See http://en.wikipedia.org/wiki/IT_audit for more info.

  14. In PCI the auditor does not certify by hugetoon · · Score: 5, Informative

    After conducting an audit of a Merchant et a PSP (payement service provider), a QSA (qualified security assesor) issues a ROC (report on compliance to PCI-DSS) that is submitted du issuers (VISA, Mastercard, Amex, JCB and Discover).

    Then the issuers certify the auditee.

    An individual can not be a QSA by itself, it has to work in an organization that is qualified as well. Among other things a QSA organization has to provision a HUGE amount of cash in case it is found liable of having unduly declared an auditee compliant.

    When a breach occurs, there is an investigation and eventually it is found that the ROC was not accurate by the time of the audit in such case the QSA organization and the QSA individual are in trouble.

    BTW a certification is only for one year.

    Now the case is not about PCI-DSS but "Cardholder Information Security Program" (CISP) and the breach happened in 2005.
    Therefore I think the outcome would not have much impact on PCI program where liabilities are well defined.

    1. Re:In PCI the auditor does not certify by hemp · · Score: 3, Informative

      CISP applied to Visa only. At the time, each payment card was instituting a separate security program. Due to feed back from merchants, all of the programs were rolled into PCI.

      PCI is very similar to the original Visa CISP program.

      The standard can be found here in case anyone is wondering what all is involved in a PCI audit:
      https://www.pcisecuritystandards.org/security_standards/pci_dss.shtml/

      --
      Skip ------ See the latest from http://www.anArchyFortWorth.com
    2. Re:In PCI the auditor does not certify by kaputtfurleben · · Score: 1
    3. Re:In PCI the auditor does not certify by Anonymous Coward · · Score: 0

      Although PCI addresses security requirements for payment card processors, passing a PCI audit does not guarantee security. There has typically been a wide variance in the consistency of audits performed by PCI QSAs. There is a lot of room for interpretation of the PCI data security requirements between different QSAs and some may be more strict in their interpretation than others. The accuracy and reliability of the information obtained by QSAs is highly dependent upon their specific audit procedures. I've heard stories of QSAs using very small samples such as two retail locations our of 1,500-store chain.

      This is resulting in the PCI Security Standards Council requiring very detailed and complete assessment of network infrastructure and systems to ensure that the PCI in-scope network is truly secure and controlled. The only way to ensure this level of rigor and consistency is to use network configuration analysis tools like Athena Security's FirePAC and Verify products.

  15. What is a certification worth? by bradley13 · · Score: 5, Insightful

    The question is: does a certification have a value, or not?

    Consider an example in a different area: accounting. At the end of the year, a public corporation must have its accounts certified by an auditor. The audit essentially states that the accounts are an accurate reflection of the company's financial state - that the accountants haven't "disappeared" a few million dollars into their private accounts, or whatever.

    If the accounts turn out to be fraudulent, the auditors have failed - and it is entirely correct to sue them.

    Back to IT certifications: if the audit missed something, then it is entirely appropriate to sue the auditors. If the security breach was not due to problems the auditors should have caught (inside job, violation of established procedures, etc.), then the auditors should not be liable.

    Consider what happens if you do not hold the auditors liable: a very current example from the financial world. The ratings agencies said that derivatives based on sub-prime mortgages were top-quality, low risk investments. Screwing up a rating costs them nothing, so they gave in to political pressure and rated these derivatives too high. Had they been liable for the consequences of their ratings, they would have done a better job. At least, one would like to think so - sadly, there is no way to go back and test this hypothesis...

    --
    Enjoy life! This is not a dress rehearsal.
    1. Re:What is a certification worth? by Opportunist · · Score: 1

      Actually, a sensible standard would consider inside jobs and make them harder. Of course, if your IT administrator is the culprit you're out of luck, but nobody else should have access to data that isn't meant for him to see.

      The bigger threat is that attack vectors change, and established standards of 3+ years ago couldn't take them into consideration.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    2. Re:What is a certification worth? by Anonymous Coward · · Score: 0

      I'm not sure I agree with your statement. As an auditor, I would *never* certify that the accounts are accurate, only that the controls in place are sufficient to mitigate a material likelihood that the accounts have been tampered. No auditor, anywhere in the world or in history has the ability to certify something as safe, secure or accurate - and I will argue with any auditor who suggests that they can (I am an auditor BTW).

      As stated many times here, an audit is only a snapshot in time. There are however significant differences in the quality of audits conducted. Some are so high level as to be nothing more than a feel good exercise. Then there are deep dives that take significant amounts of time, money and resources, which offer a better quality result. My company uses several external auditors, and I have literally thrown some audits in the garbage (technically a secure shredder) after reading them due to the fact that they were crap. As an example, I had an auditor explain to me that having the HTTP service installed on a server was a significant vulnerability that needed to be uninstalled. I wouldn't have argued so much had the server in question been a web server.

    3. Re:What is a certification worth? by Anonymous Coward · · Score: 0

      Just to correct the above comment about accounting/financial audits....

      Financial statement audits (FSA) do NOT state the accounts are an accurate reflection if the company's financial state. Nor are they a 'certification.' They are an opinion that provides reasonable assurance that the financial statement and free of material misstatement, whether caused by error or fraud. Most types of fraud are very, very difficult to detect in a FSA. A full forensic audit would be required to give a higher level of assurance.

      Most fraud is is caused by collusion. Collusion is nearly impossible to detect during an FSA. Also, several million dollars may be missing, but that amount would not be material to any large corporation. Immaterial amounts would not even be subject to the auditor's inspection.

  16. Interestingly, PCI-DSS does not itself... by tlambert · · Score: 1

    Interestingly, PCI-DSS does not itself appear to be sufficient to prevent a security breach in the first place; among other things, they mandate a set of principles which are pretty, but not a guarantee:

            https://www.pcisecuritystandards.org/security_standards/pci_dss.shtml

    Yeah, if everything were to work out, and all virus threads were already covered by the antivirus software, and there were no such thing as a zero-day exploits, it might stop a penetration. But not otherwise.

    -- Terry

  17. Erh... I thought that was the point? by Opportunist · · Score: 1

    Seriously. What else are certificates good for? If it's just "drop some money so we send you a guy that hands you a cert", what does the certificate mean? I mean, besides "we had enough money to buy it"?

    Certificates are worthless if they don't certify anything but having enough money to have an auditor squat at your company for a few days. And if auditors are not liable for the validity of a cert, that's basically all they really prove. Why else should the auditor really audit a company and not just hang out there and surf for porn for a few days so it seems he did his job?

    --
    We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    1. Re:Erh... I thought that was the point? by Aladrin · · Score: 2, Interesting

      Exactly. A certificate -certifies- something. If it doesn't, it's not a certificate.

      The real question here is: What should happen to the certifier if their certificate proves false.

      I don't think this is a government question. If there's nothing in the contract about this scenario, then you paid for -nothing-. And if there is, you already know the solution to the problem... It's right in the contract.

      --
      "If you make people think they're thinking, they'll love you; But if you really make them think, they'll hate you." - DM
    2. Re:Erh... I thought that was the point? by Opportunist · · Score: 1

      If the cert was issued when the requirements were not met, the certifyer should be liable for it. Simple as that.

      I've been in more audits than I really wanted to stomach. Usually, they're a weak joke. The reason for this is very, very simple: Auditors want to audit other companies too, and companies talk with each other and "recommend" auditors based on their experience.

      Question for 500: What will get you recommended? A through audit, painstakingly dragging out every piece of possible dirt and repeating the audit 3 times because some points were not met, or passing a company easily even if they flunk here or there? Remember, companies want certificates because they have to, to get contracts or to comply with legal requirements, they do not want certs because they want to improve themselves.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
  18. Here's how it'll go: by iCEBaLM · · Score: 1

    Plaintiff: "Your Honor, we're suing defendant because they certified our credit card system as being PCI compliant, yet it was breached by hackers."

    Judge to Defendant: "Is this true?"

    Defendant: "Yes Your Honor, however being PCI compliant does not guarantee you will never be breached by hackers."

    Judge: "Case dismissed!"

    1. Re:Here's how it'll go: by RayMarron · · Score: 1

      You beat me to it! The auditor only says your setup meets a specification. It does NOT guarantee that implementing the specification will keep you safe from all harm.

      --
      ON DELETE CASCADE
  19. Question by Loki_666 · · Score: 1

    Who audits the auditors?

    1. Re:Question by Anonymous Coward · · Score: 0

      Other auditors. No, seriously. When the auditors need to be audited, they call the competition. Easy as that.

    2. Re:Question by Maximum+Prophet · · Score: 1

      Who audits the auditors?

      The auditor's insurance companies. They are the ones that will have to pay if the auditor loses the lawsuit.

      --
      All ideas^H^H^H^H^Hprocesses in this post are Patent Pending. (as well as the process of patenting all postings)
    3. Re:Question by gmack · · Score: 1

      PCI does. All PCI audtors have to be retrained each year.

    4. Re:Question by Opportunist · · Score: 1

      They don't audit. They assess risks. How likely is it that the company the auditor audited gets hacked? Even assuming the auditor is using his auditing time for surfing porn, it's anything but 100%. Not every company gets hacked, and thus the certificate "holds".

      That the cert is utterly worthless is not really of interest.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    5. Re:Question by Maximum+Prophet · · Score: 1

      Before the meltdown, I would have argued with you. In order to assess risk, insurance companies need to understand the business and do the same things an auditor would do. After the meltdown it was found that the insurance companies just sold a lot of insurance, their execs got bonuses, and when things crashed, they went bankrupt (or sought a bailout)

      --
      All ideas^H^H^H^H^Hprocesses in this post are Patent Pending. (as well as the process of patenting all postings)
    6. Re:Question by Opportunist · · Score: 1

      They don't gauge the risk of something failing. They assess the risk that they have to pay. Simple as that.

      I've spent my time in the banking and insurance branch. Did you know that suicide rates and vis major are usually on the "beneficial outcome" side of the risk assessment of life insurance risk assessment sheets because the insurance won't have to pay?

      Insurances don't care how likely it is that something bad happens to you. They care how likely it is that they get to pay for it. And the same applies to auditors. It's not how likely it is that the auditor blew the audit, it's how likely it is that it will be discovered.

      --
      We used to have a Bill of Rights. Now, with the rights gone, all we have left is the bill.
    7. Re:Question by jvkjvk · · Score: 1

      Did you know that suicide rates and vis major are usually on the "beneficial outcome" side of the risk assessment of life insurance ...

      So, it is in the best interest of insurance companies to promote suicides?

      That would explain a lot.

  20. Audit Responsibility - Possibly a good thing. by luftmatraze · · Score: 3, Informative

    I am working in a large firm. Quite often new projects upon realisation require technical audits as well as "Life Cycle" audits for existing systems involved with billing etc. One point that needs to be clear. Audits are not cheap! These guys are paid between 1500-2000 per Man day. Presently this is done in essence without ANY liability as to the quality of their work. What needs to be established in this case is: 1. Technical Audits provide a snapshot of a system "at a particular point in time" - Did at the time of the Audit these holes exist, or where there changes afterwards which could have affected the audit results? 2. Audit Scope. This is really important! If the Audit scope didn't include for instance the visibility of the systems from outside of the firewall, then the perspective of the auditors were limited and therefore the audit itself is not complete. I have seen companies for instance that are ISO 27001 Certified....however.... the audit scope was only for a particular part of the company. This enables the company to suggest 27001 Certification when in fact it may not indeed be fully the case. Most likely the outcome of such a case would be an increase in costs to cover Liability (insurance or something of the like) on the part of the auditor. However it may well be also an increase in the quality and transparency (clearer scope, limitations etc.) of technical audit work. Both of these are positive outcomes! http://streetstyles.ch/ - Swiss Band & Fashion Tshirts

  21. That is a bad idea - very bad indeed by Anonymous Coward · · Score: 0

    First off, an auditor can only confirm a PRESENT situation, not a future one. In other words, that they do the right thing when the auditor is "in da house" doesn't mean they'll do that when he/she has left.

    Secondly, I'm rather averse to the whole money game, because this is where Andersen came unstuck. Creating liabilities ensures that only the big boys can play, and only the big players can afford an audit unless compelled by law. I much prefer a system where a certain volume of independent auditors exists because it makes collusion so much harder.

    Thirdly, allowing any entity to quickly blame their auditors when things blow up means an escape route for management: spend less and plan to sue the auditor when it blows up. Very Much Not Good IMHO.

  22. Follow-up yo Obama's speech to Muslims - Fuck you! by Anonymous Coward · · Score: 0

    Stop acting like spoiled little children - like it's the rest of the world who has to respect your religion and not the other way around. Stop systematically oppressing non-muslims in your midst, and stop threatening apostates with murder. Stop beating, raping, and gouging out the clitorises of your women. Stop murdering homosexuals, and stoning people who say things you don't like. Stop murdering critics. Stop burning cars when someone draws an ironic cartoon of your silly pedophile war-mongering "prophet." Stop denying the Holocaust, an event for which living witnesses still exist and which has been exhaustively documented. Stop your backwards, fucked-up, middle-ages, barbaric bullshit. Yes, basically I'm telling you to give up Islam, which glorifies all the things I just mentioned.

  23. The whole concept of this question is stupid... by Anonymous Coward · · Score: 1, Informative

    I audit IT Security for a living, and have just finished a Level 1 PCI-DSS study.

    An audit does NOT "certify a system as secure". It certifies that certain features of the system are present and working. A computer system (or any other system, mecanical, electrical, human based, or what-have-you) may fail for a variety of reasons.

    The features which are intended to prevent failure may break, or not work, or be irrelevant to the actual failure mode which occurs.

    If a system fails, having had it audited may help to prove that you were doing all that was reasonable to protect against failure, but it does NOT mean that it will not fail.

    The only reason for suing auditors would be if they did not provide the audit service that they claimed. If I audited a system as having an anti-virus package and it did not have one, I could be sued for failing to audit properly. You might find out that my audit was incompetent if, after I had certificated it, the system failed with a computer virus that the missing AV package should have picked up. But you cannot sue me for the impact that virus had. You can only sue me for not doing my work properly. I will NOT have claimed that the system was secure. I will have claimed that it was running X Anti-Virus, and if you can show it was not, you may claim the cost of the audit back. But you cannot then charge the cost of the clean-up to me....

    1. Re:The whole concept of this question is stupid... by 91degrees · · Score: 1

      But you cannot sue me for the impact that virus had.

      Sorry, but why not? This is exactly the sort of problem that you were paid to prevent. The company proceeded with the understanding that this would not happen as a direct result of your actions. They paid you to take responsibility for their compliance. If you fail to ensure that the system was compliant then it's your responsibility.

  24. Compliance != Security!! by Anonymous Coward · · Score: 0

    I do security assessments for companies. Management needs to realize that simply being compliant isn't enough to be secure. Compliance is a good starting point, but you can't just say "Look, we're compliant with X standard." and pretend like nobody could ever hack in, or no user could ever screw up. There have been breaches where the company breached *was* PCI compliant. Being compliant doesn't mean what most people think it means.

    1. Re:Compliance != Security!! by bluefoxlucid · · Score: 1

      I was just about to post this. PCI-DSS says you need a "firewall" in front of your database server. It doesn't say the firewall should block anything.

    2. Re:Compliance != Security!! by 91degrees · · Score: 1

      It comes down to whether they were PCI compliant or not. Simplifying hugely, if a security requirement is "All doors have locks", and the doors do have locks then you can tick that box and gain the certification. If the doors are left unlocked, then that's a process outside of the certification and so you're not responsible.

      If you ignored a door and figured most of the doors having locks was good enough then you are responsible.

      So, yes you're right. It's just that it's possible they were certified as compliant when they weren't.

  25. Re:Interestingly, PCI-DSS does not itself... by gmack · · Score: 1

    That's all PCI is really: a bunch of pretty looking standards.

    The upshot of all of those standards is that I now have a whole set of servers that I absolutely hate to work on.

  26. the bank pokes the wrong pile by Stumbles · · Score: 0

    Sounds like someone is looking for a scapegoat. No, the auditors should not be held liable for a standard they have no control over defining. That blame rests with the PCI Security Standards Council and the Federal government who define the contents of SOX. It is already well known the standards for security set by PCI-DSS is laughable. There was a youtube about such from England, and I am sure there are other documents that confirm just how bad their standards are. The biggest problem with PCI-DSS is their firm belief that denial of the problems pointed out in that video works just as well as security by obscurity. So is this the name of the game now... sue people FOR following the standards, even when they have no control over them?

    --
    My karma is not a Chameleon.
  27. Re:Follow-up yo Obama's speech to Muslims - Fuck y by Anonymous Coward · · Score: 0

    And P.S. - stop using the Israeli/Palestinian issue as a red herring to distract from your own leaders who have failed you and ensured that you would be left behind by the rest of civilization by their self-serving actions. And don't make the mistake of thinking that Obama actually cares, or that he intends to keep his word. The only thing he cares about is cultivating his global cult-of-personality image.

  28. 3rd party negligence by QX-Mat · · Score: 1

    As a 3rd party, auditors remarks and certifications can give a representation inducing someone to wrongfully contract. There is an established principle in the tort of negligence that allows an injured claimant to sue a non-contractual 3rd party in this type of misstatement. We call this the Hedley Bryne principle (google it).

    It's great. Providing you can establish a duty via special relation ship of reliance.

    It's bad for contracting parties however, if you imply that all contracts should have this principle without the need for a special relationship of reliance, because priciple parties (the other guy, not the certifing agent) cock up - and it's their fault. It should be their fault. How am I to control the actions of someone I *thought* was competant?

  29. They Aren't Liable Now?! by DeanFox · · Score: 3, Insightful


    A Notary Public can be held responsible but an auditing firm isn't? I would have thought they already were held liable. If they're not, what a great job! Like a Notary Public that can stamp, validate and vouch for anything without cause for concern. It's probably because the Notary is people. The auditors are corporations. Corporations are just like people absent accountability or morals. Corporations are like Sociopaths. And as they're running the show, corporations are like Sociopaths in an Anarchy.

    -[d]-

    1. Re:They Aren't Liable Now?! by herske · · Score: 1

      Where I live, a notary public certification does not mean "truth" in an absolute sense. A notary public only certifies the authenticity of representations made by a third party.

  30. It will just raise the prices by C_Kode · · Score: 1

    It will just raise the prices since the auditors will have to take out insurance. Of course, what are banks to do? They hire someone to ensure they are compliant, then get screwed because they were compromised by something that is included in the compliancy!

    Security is an imperfect world. I'm sure something will come up that a company was compliant, but still gets compromised and attempts to sue the auditor. Then again, I've met more than a few auditors that had no busy being in the security business!

  31. This is a positive development by oldbamboo · · Score: 1

    Very much in agreement.
    I spent some time in IT audit for one of the Big 4, and it's always puzzled me that they can issue a draft audit point which if challenged is just taken away. If accepted, lots of monkeys have to run around at great expense clearing it. It seems a bit rich to me that there is no penalty on the auditor for this. effectively they can just rain paper with little consequence, and at potentially huge cost to the client.

    Having said that, these firms are partnerships, there is always a partner very close to the work being undertaken, and it's their ass and their money and as a consequence the QA at these firms on their deliverables was exceptional in my experience.

    But this is an issue, and I think that legal redress is deperately needed.

    To illustrate this, I recall one audit I had to do. It was a follow on from the previous years IT audit a colleague had done for one of the two biggest banks in the country in question. One of the previous years recommendations, signed off on by the business, was the need for Network Intrusion Detection to be put in place. This was actioned, and when I got there they had had an expert working day in day out for months, with a huge budget for some very expensive network taps and headcount for monitoring. I reviewed the point, determined that they hadnt yet implemented the control as of that date, recommending that they proceed and introduce it within the coming year.

    At the close out meeting one of the commercial directors ate us alive. The original point should never have been accepted. The banking industry, at that time, hadnt settled on NIDS as a requirement and host based should have been fine. Effectively our sloppy report made them piss millions up the wall for little reason.

    Audit reports are clear documents, beautifully built, well evidenced. They always have work papers and test papers behind them. They are perfect candidates for for further inspection in a court of law and I have seen, first hand, instances where they have been harmful and inaccurate and should be subject to this scrutiny. If a process or test was missed off, it will show. Every time.

    Yes, it's true that senior management at the bank signed off on the previous years report, but this was in good faith that my firm knew what they were talking about. They didnt, and should have been liable. Why not? Currently they get out of jail if they're right, and they get out of jail if they're wrong. And dont even get my started on the conflicts of interests I saw!

    --
    You may not agree with what I say, but you should fight to the death to allow me to say it, by modding me up.
  32. PCI is not 100% safe by defenition. by ACMENEWSLLC · · Score: 1

    PCI in itself doesn't guarantee 100% safety.

    It says so right on their common myths page;
    https://www.pcisecuritystandards.org/pdfs/pciscc_ten_common_myths.pdf
    Quote "Successful completion of a system scan or assesssment for PCI is but a snapshot in time.
    Security exploits are non-stop and get stronger every day, which is why PCI compliance efforts
    must be a continuous process of assessment and remediation to ensure safety of cardholder
    data."

    It's a very good PDF to read. Now if the auditor said they were PCI Compliant and there was something obvious he negligently overlooked, that would be another thing. But PCI does not mean you can not have a data breech. You can never be 100% protected from that. What if someone came in, put a gun to a privileged employees head, said give me all your data or he dies? Will PCI stop him?

     

  33. Re:Interestingly, PCI-DSS does not itself... by Anonymous Coward · · Score: 0

    You can call them "pretty looking standards", some others might call them "minimum security requirements based on a risk-based security management approach". There is no such thing as 100% secure, nor is security some goal that when achieved it can be forgotten. Security measures try to improve security to a reasonable level using available resources to minimize risks. It's a process, that in order to remain effective has to evolve and continuously re-evaluate itself.

    PCI SSC aka the cars brands have several controls in place to make sure the auditors do their job properly and ethically:

    - Audits can only be performed by organizations that have the QSAP status
    - Audits can only be performed by trained and certified QSAs, who have to recertify annually
    - Submitted PCI on-site audit reports (RoC) are scored by the SSC, anything less than a near-perfect score can lead to warnings or loss of certification status
    - In case a PCI certified entity is breached, the SSC will conduct a forensic investigation. If the certified entity is found to be non-compliant and the QSA has failed to fully validate its compliance, also the QSA can face penalties.

    I've heard many stories about QSAs giving out certifications on false grounds, or even fraudulently, hopefully these will be taken care of to retain the credibility of the QSAs and the PCI.

  34. You don't understand what "certification" means by TrueKonrads · · Score: 2, Informative

    I am an IT auditor working for a company that You would call if You would want to be certified.

    Certification means that there is a work (audit) programme that states control objectives. Auditor follows this programme very closely and then, if the issues are within some zone of tolerance (which may be zero as well), auditor writes a statement that company XYZ is compliant with this and that.

    What it does NOT mean is:
      a) a certified company will follow its practice after certification (they may just have put a convincing show).
      b) that there are no other issues with the company that are outside of work programme
      c) that sysadmin will be dilligent in future to apply timely patches

    A PCI-DSS compliance says "There are no critical issues on the surface". That's it.

    --
    Lone Gunmen crew.
    1. Re:You don't understand what "certification" means by mkettler · · Score: 1

      I'd agree. Auditors are not a be-all end-all. It is not the auditors job to detect all unknowns about your company, they go on what you provide them, and the amount of digging they do beyond that is modest. Deep digging is the job of your staff, not auditors.

      However, I do think there should be liability for items clearly in-scope.

      For example, if I hire an auditor to audit me for PCI-DSS, I expect they will at least review all of the basic requirements of that standard. If an auditor blatantly neglects to investigate or even mention an item spelled out in the standard, they've fraudulently issued that certification. ie: if a PCI-DSS auditor never even asked if the network has firewalls, and just assumed it did, that's the auditor being negligent. Auditors are there to ask questions, review the answers and verify compliance, not make assumptions.

      But baring clear and substantial negligence by the auditor... Good luck with that.

      --
      -Matt
  35. Sued for what exactly? by Klync · · Score: 2, Insightful

    I'm surprised nobody mentioned this yet: adherence to PCI-DSS does not necessarily guarantee that your system cannot be cracked or broken into. PCI-DSS provides a set of guidelines - created by the banks and cc companies themselves - which must be met in order to be considered safe enough to be allowed to process transactions. Now, if the auditor was negligent or deceptive in certifying the system as compliant, this seems like a no-brainer lawsuit. However, it is entirely possible that the system *was* compliant, but got cracked anyway.

    --

    ----
    Not to be confused with Col.
  36. Certified as secure by rcamans · · Score: 1

    The auditor did not certify them as secure. They certified them as PCI-DSS compliant. That just means that they are somewhat hard to penetrate. A certain amount of time, expertise, and tools limit is in the standard. Any criminal who spends more time, money, expertise to successfully penetrate them is ok by the PCI-DSS standard. The standard actually explicitly says this stuff.. PCI-DSS is just setting a bar. Hopefully one high enough that common criminals cannot easily or quickly beat it. But organized crime (Mafia, Russian Mafia) can afford to buy the expertise, spend the time and money. Or a common criminal (script kiddie, haxor boy) can spend enough time and defeat it as well.
    I know. I designed a PCI-DSS compliant system which was certified.
    So, no matter what the bank says, if the other guys were actually PCI-DSS compliant, the bank loses the court case. Unless, of course, they buy better legal representation. Since the court system is biased towards the rich and mighty, who can afford better legal representation.
    And any case can be won by a good enough lawyer.
    For example, a lawyer won that a board fell off a store shelf and took away the complainant's psychic powers.

    --
    wake up and hold your nose
  37. Security is the responsibility of the company by NetNinja · · Score: 1

    I have just gone through a PCI audit to get my company certified PCI level 1.

    It may be the auditors job to throughly go through your systems but it also up to you to honestly provide those answers to his questions. He will of course verify your answers.

    Then it is the security Admins job to ensure that the network or data is secure.

    Just because you have passed an audit or got the latest doo hicky to monitor your network, it is up to you to review the logs and to ensure on a dailiy basis that YOU were doing your job.

    I think this is a case of someone trying to save thier job.

  38. Car by fulldecent · · Score: 0, Troll

    I take my car to PepBoys for a yearly inspection in January. If my brakes go bad in February and my rotors are worn, that's PepBoys fault and they fix it.

    --

    -- I was raised on the command line, bitch

  39. lolol by dstones · · Score: 1

    Just because you are compliant with some regulation does not mean that your system cannot be breached. That would be absurd to believe. Sure PCI is a regulation affirming that the system is secure, but secure is defined by their set of regulations. You cannot protect against everything. I think it's a joke to hold the auditor responsible, unless like some others have stated, that it is possible to prove that the job done was insufficient.

  40. Not just costs, 100% of all audits will result... by Anonymous Coward · · Score: 0

    ...in an automatic failing score from that point onwards. There will cease to exist the possibility of achieving a passing grade on any systems security audits anymore. If the bank wins this lawsuit, then the only thing that such audits will achieve from that that point forward is reports showing varying degrees of security failure and risks.

    And perhaps that's the way it should be anyway, because in truth there is no such thing as a secure system that involves information technology and humans.

  41. Messy business anyway ... by Anonymous Coward · · Score: 0

    I guess an auditor who does not specify exactly the "scope, assumptions and disclaimers" of his certification is not worth his title in the first place. Professionally speaking : if you are auditing "the bridge" as in the examples mentioned, you would have either : a) clearly stated that the quality of steel used is not in scope, OR better still, taken a random sample from an un-critical segment and tested it in a lab, or something to that effect. In any way taken, its a very risky business, because even if you state your "disclaimers" in the contract, and something does happen you are still exposed on an ethical or reputation level, because not everybody will have read your contract - and what the relevant public would still say is that XYZ audited and certified a bridge that fell.

  42. Wait they aren't already? by nurb432 · · Score: 1

    If they are not liable for their mistakes in certifying then what value is their approvals?

    Sure, if you lie to them its your fraud and its not their fault, but if they make the mistake...

    --
    ---- Booth was a patriot ----
  43. This was an eye-opener for me by Punk+CPA · · Score: 1
    Payment Card Industry Data Security Standards seem kind of weak to me. Here are just some of the issues:
    • Independence PCI DSS auditors are permitted to audit companies where the auditor sold, installed, configured, or has rights to the security software being used. Also, if the auditor disagrees with the client, the client is free to hire a more pliable auditor with no one the wiser.
    • Scope The standards permit the client to limit the scope of the audit to defined systems and their components using defined methods. If the client doesn't want to pay for penetration tests, the auditor doesn't do them.
    • Completeness A typical PCI DSS audit uses the client's system and security documentation as the starting point. The responsibility for gathering other evidence is limited. There is no requirement to do any network scanning (like with NMAP) or to go sniffing for undocumented wireless entry points, so there may be elements of the system not documented and not tested. This sounds like the case discussed here.
    • Validation PCI DSS auditors are not responsible for verifying that the client's controls worked as intended. There is no mandate for penetration testing, war driving, or independent virus scanning.

    Even if the auditor had done his job (not really clear from the articles), that to me would not demonstrate that the customer data was safe.

    Links:
    Congress is not happy, either.
    PCI DSS Validation Standards
    PCI DSS audit procedures

    So much for my lunch break.

  44. They should be held responsible by onyxruby · · Score: 1

    Audit works has been about 20% of my workload over the last few years. Auditing isn't about having the perfect environment (which I've never yet seen), it's about being able to say "I have conducted business in a good faith manner following industry best practices" - and that is what allows you to win in court. When management brings you in for an audit they are expecting someone to find these kinds of problems and point them out. They need someone who is /not/ a staff member, has no stake in things, no political ax to grind to come in and verify that things really are OK. I've seen environments like the client I'm with now that went years without an outside auditor before I came in and these are typically the ones that you hear about on the news for massive breaches.

    Auditing is about trust and the reassurance that your systems are running under industry best practices and do not have undocumented security risks. Often times it takes an outside auditors report to get through red tape so that budget /can/ be allocated. Management (and it's not uncommon for audits to be paid for outside of IT's budget) needs to have something that they can trust and that they can use to have a legally defensible position. The auditors job is to find holes, identify problems, explicitly identify risk, review personal and so on and then document it. That being said, the auditor always runs the risk of being asked to fix what they find, so the auditor needs to be realistic in their work.

    Insurance policies, industry certifications, millions in losses and public goodwill all ride on these reports. Some auditors are afraid of writing a critical report as they fear they will personally be poorly reviewed by the client if they do, or they do not want to risk offending the client and losing repeat business. This is where lawsuits come in, so that the integrity of the audit is placed before fear of losing repeat business. That being said, writing reports that tell a client they don't know jack and have to redo everything and that they should hire some additional personnel without offending anyone is an art form if it's own sake.

  45. This is a perfect example of why Wiki sucks balls. by HornWumpus · · Score: 2, Informative

    Damn Wikipedea sucks balls.

    Some moron gets it into his head that the Tacoma Narrows bridge failed due to 'aeroelastic flutter' not resonance. The definition of 'aeroelastic flutter' begins with the description:

    Flutter is a self-feeding and potentially destructive vibration where aerodynamic forces on an object couple with a structure's natural mode of vibration to produce rapid periodic motion. Flutter can occur in any object within a strong fluid flow, under the conditions that a positive feedback occurs between the structure's natural vibration and the aerodynamic forces.

    Emphasis mine

    In any case the bridge was visibly in resonance torquing in its second harmonic. WTF do you think 'natural vibration' means.

    The editor of the Wiki article goes to great lengths to prove he doesn't really know what resonance means. He quotes some profs point that there wasn't resonance between the vortex shedding and the natural frequency (something started it torquing, ). Completely missing the point that flutter is still resonance.

    --
    John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'
  46. How do we work this into our legal system? by sorak · · Score: 1

    It seems to make sense that, this should be treated like most other audits. If the auditor failed to notice a problem that it would be reasonable to assume a professional would get right, then he should be accountable. But this brings the same problem we have with IT patents: How do we fix our legal system so that the authorities are qualified to weed out the BS in complicated technical matters?

    I'm starting to think we need a separate court system just to handle technology-related cases. One in which it is reasonable to say "If you can't explain DNS", then you're not qualified. But that's just a thought...

  47. This just in: by Anonymous Coward · · Score: 0

    Being PCI compliant does not make you invulnerable, it just means that you meet PCI standards. PCI-compliant organizations get compromised on a regular basis. The auditor isn't auditing to say "this organization is secure". They are auditing to say "this organization met the requirements of PCI standards compliance at the time the audit was conducted". That's a HUGE difference, and the point where liability effectively is shifted back from the auditor to the auditee.

  48. Re:Follow-up yo Obama's speech to Muslims - Fuck y by Anonymous Coward · · Score: 0

    What a relief that Obama is about three times as intelligent as George Bush. And, about four times as intelligent as the poster child above. Where Bush promised a crusade, Obama salaam'd Islam. Winning hearts and souls - the right way. Now, it's time for the parent poster to bite a big salaami ...

  49. Two names, three words by Nefarious+Wheel · · Score: 1

    Arthur Anderson. Enron.

    --
    Do not mock my vision of impractical footwear
  50. security is a process by Anonymous Coward · · Score: 0

    Being certified compliant with the PCI standards does not ensure that your networks and systems will never be compromised. Data security is not about ticking off items from a checklist. Security is a process of managing your risks and minimizing your losses. Stricter compliance with the PCI requirements may reduce your risk, but certainly won't eliminate it.

    The recent massive data breach at Heartland Payment Systems is a case in point. HPA was certified PCI compliant before the breach was discovered. In this case, the attack involved custom malware designed to capture data.

    That being said, there has been a wide variation in the quality of QSAs. The PCI Security Standards Council is trying to improve accuracy and reliability among QSAs. They are going to be held responsible for much more detailed analysis of network infrastructure. Even for small networks, a manual assessment of the network device configurations will never be sufficient to ensure that the PCI in-scope network is secure and controlled. We can expect to see QSAs increasingly relying on network configuration analysis tools such as Athena Security's FirePAC and Verify to ensure completeness and accuracy in their assessments.

  51. security is a process by Anonymous Coward · · Score: 0

    Being certified compliant with the PCI standards does not ensure that your networks and systems will never be compromised. Data security is not about ticking off items from a checklist. Security is a process of managing your risks and minimizing your losses. Stricter compliance with the PCI requirements may reduce your risk, but certainly won't eliminate it.

    The recent massive data breach at Heartland Payment Systems is a case in point. HPA was certified PCI compliant before the breach was discovered. In this case, the attack involved custom malware designed to capture data.

    That being said, there has been a wide variation in the quality of QSAs. The PCI Security Standards Council is trying to improve accuracy and reliability among QSAs. They are going to be held responsible for much more detailed analysis of network infrastructure. Even for small networks, a manual assessment of the network device configurations will never be sufficient to ensure that the PCI in-scope network is secure and controlled. We can expect to see QSAs increasingly relying on network configuration analysis tools such as Athena Security's FirePAC and Verify to ensure completeness and accuracy in their assessments.

  52. As a former auditor by blippy · · Score: 1
    Many years ago I worked as an accountant in the UK. I have never been involved in SOX, but can offer a perspective on audit reports.

    First off, an audit report is for the benefit of the shareholders, not the management. Management prepares accounts, and auditor signs it off as representing a true and fair view of the financial position of the company. This gives the shareholders some confidence that the figures aren't just totally made up.

    Secondly, it is managements responsibility to manage the company. Not the auditors. It is up to management to put in everything in place that needs to be put in place, and ensure that everything is working correctly. This is what it means to be a manager. The auditor merely counts the beans and ticks the boxes.

    Thirdly, auditors do not owe a duty of care to the company. When performing an audit, they should conduct their work neither to expect malfeasance, nor neglect it as a possibility. However, if they have reason to believe that there is malfeasance, then it is their duty to perform a proper investigation.

    Fourthly, auditors usually write a report to management explaining where their accounting procedures could be improved. Management often dismisses such recomendations. Imagine, then, the scenario where the auditor has to be accountable for what essentially boils down to the actions of the management. They'd presumably write book-length reports on what needs to be done. If anything were to then subsequently go wrong, you'd have lawyers pouring over these reports with fine-tooth combs. The auditors lawyers will be asking management "did you implement this recomendation?", "did you implement that recomendation?", "how about this one over here?". "No? Oh sorry, we can't be held responsible when we've clearly laid out the defects, and you refused to correct them."

  53. Re:This is a perfect example of why Wiki sucks bal by Ihlosi · · Score: 1
    In any case the bridge was visibly in resonance torquing in its second harmonic.

    No. The way it was vibrating was the _shape_ of the second torsional mode (not harmonic), but not the _frequency_ of the second torsional mode of the bridge. In case of resonance, you would see both the shape and the frequency of the mode in question.

  54. And the question is ... by DrYak · · Score: 1

    PCI covers more than just servers ---- it covers physical security, staff identification, physical access to paperwork, disposal, data retention, lots of corporate policies.......

    Was the Bank's security breached through exactly one of those things ?
    When the breach happened, did they strictly follow the procedure precisely as it was certified ?

    If something changed :
    - New (non-secure) components have been added to the system and those were the entry points.
    - New bugs, new security holes or other have been discovered that weren't know at the time of certification.
    Then, it is NOT the fault of the auditors who gave the certification.

    If nothing had changed :
    - The bank strictly kept everything exactly as certified. But nonetheless one of the supposedly "certified secure" element got broken through, because it wasn't actually secure in the first place and got overlooked by the certification
    Then it IS the fault of the auditors who gave the certification.

    Indeed the world of security is complex and you can't certify that something will be universally secure against any possible threat current or future.

    BUT if you give an XYZ certification, the system should be secure against any threat taken into account by XYZ at the time of certification.

    To give a concrete example :

    physical access to paperwork, disposal

    For exemple, if at the time of certification, PCI required that paperwork be burned by company's own trusted employee, but the bank had regularly some of the critical papers nonetheless finish in the paper-recycling bin, it's the auditor fault. They gave certification even though the bank didn't follow proper procedures.

    But if the bank had recently switched to use shredder (a not certified model), or if the physical access of the paperwork is protected inside a vault, whose mechanism is discovered to be flawed since the last certification project, it's not the auditors fault. They gave certifications according to some set of rules. If the bank subsequently fucks up, or if new data has arrived in between, it's not the auditor's fault.
    That's also why auditing should be done on a regular basis.

    --
    "Sufficiently advanced satire is indistinguishable from reality." - [Tips: 1DrYakQDKCQ6y52z6QbnkxHXAocMZJE61o ]
  55. Re:This is a perfect example of why Wiki sucks bal by HornWumpus · · Score: 1

    It was in the second frequency.

    Bridges don't vibrate at other then their natural frequencies any more then guitar strings do. (That is to say they do, but the energy dissipates quickly.)

    As to the claimed natural frequency of the second torsional harmonic I'll say citation needed. Who am I going to believe you or my lying eyes.

    --
    John McAfee 'It was like that time I hired that Bangkok prostitute; to do my taxes, while I fucked my accountant'