Questioning Security Certifications

prostoalex writes "BusinessWeek questions the validity of security certifications in the modern world. They take a look at Federal Information Processing Standard and the certification process. Apparently 'the testing companies make money by certifying products, not catching problems' thus implying that the seal of approval might not mean a whole lot."

Comments

NIMDA?

[peterdaly]

Automated software is a good baseline approach, but it falls far short of cunning humans hammering away at systems.

Then why is once an hour does my apache webserver have clients trying to access dll's in the log files? I am sure the IIS admins may not agree with that statement.

-Pete

According the the Orange Book..

[Dynamoo]

According to the Orange Book, the now-slightly-obsolete DoD certification, Windows NT 4.0 is secure enough to get a C2 Certification.

Now, before we all laugh and say "doesn't it show that the certifications are stupid?" consider this.. maybe the certification system does work, and all those other certified products are equally flaky. I've got a list of some TCSEC-certified systems here and frankly it's a pretty unappealing set of OSes. If there were as many Unicos systems (rated B1) out there as there were Windows, I betcha they'd find holes in it soon enough. The fundamental problem with any popular OS is that there will be thousands of hackers and wannabees probing away at it. I don't think there are many people reverse engineering CA-ACF2 MVS in their bedrooms.

I think the motto should be: "Security Through Obscurity" - perhaps all those horrid proprietry OSes did have a point after all.

Re:Certifications of any Type

[Iorek]

I agree that a certification isn't worth much if you don't know what it means. I think that's a big advantage of the Common Criteria (CC). While it's a lot to digest, there are guides out there to help you through it.

I work for the Canadian Common Criteria Scheme and it's my job to ensure that the Canadian labs follow the CC correctly and consistently in their evaluations. I found the article invaluable and disturbing (especially the Bruce Schneier quote), since we're obviously looking for ways to promote the CC, and the article highlights the concerns we need to address.

"unbreakable" oracle has 15 certs

[kirkb]

At http://www.oracle.com/ip/deploy/database/oracle9i/ you will see:

Oracle now holds 15 security evaluations. DB2 has none. SQL Server has only one.

If it was easy to "buy" these certifications, I'm sure that Microsoft SQL server would have more than just one by now. (Granted, Oracle also has a bit of cash to throw around too).

Software changes too fast

[SiliconEntity]

I was involved of getting our software package FIPS 140 certified, which is the major crypto security certification. I think there's some validity to the point that the certification house (which is sort of a gatekeeper to the actual certification) has something of a conflict of interest. We are paying them for the certification, while they are the ones who check the adequacy of our security measures. FIPS is supposed to check on their work, but that was largely a rubber stamp.

Nevertheless the certification house did do a thorough check on us and did recommend a number of changes to our software. We didn't think any of them truly added security, but at least this way it was obvious that the cert company was doing their job.

The big problem is that we got that version of the software certified, taking about eight months and several employees' time. Now a few months later we come out with a new release! We can't get re-certified every time, even though they have a shortcut for recertifications. Keeping up with the short software release cycle would be way too expensive.

So we still have FIPS 140 certification listed as a feature of our product, but if a customer really wants that specific version, we have to sell him old software. As it turns out, no one does. All they really need is to be able to check the box that says we are certified, and then they're perfectly happy to take the latest software. The mere fact that we spent the time, effort and money to be certified is what really counts.

Patents Office...

[Julz]

Sounds like the same problem with the US Patents Office.

They should pay them on the number of patents thrown out.