Slashdot Mirror
Questioning Security Certifications
prostoalex writes "BusinessWeek questions the validity of security certifications in the modern world. They take a look at Federal Information Processing Standard and the certification process. Apparently 'the testing companies make money by certifying products, not catching problems' thus implying that the seal of approval might not mean a whole lot."
There are plenty of industries where following the money would make you think twice about the motivations of the seller of services. How about financial planners/brokers who make more cash by churning your investments? How about the auto mechanic who makes more cash by replacing your radiator when all it needed was an external cleaning (true personal experience here)? How about [fill in your own example here...everyone has one]?
At this point, if you're not always questioning whether a service provider is taking you for a ride, then you're being taken for a ride.
How can we afford to ever sleep
So sound again
--ebtg
Automated software is a good baseline approach, but it falls far short of cunning humans hammering away at systems.
Then why is once an hour does my apache webserver have clients trying to access dll's in the log files? I am sure the IIS admins may not agree with that statement.
-Pete
Soccer Goal Plans
--Automated software is a good baseline approach, but it falls far short of cunning humans hammering away at systems.
:
:
Automated software cuts costs. That's why they use it. Human security testers are expensive, even though IMHO it might be a good way for the most talented script kiddies to make a buck during summer...
--The testing companies make money by certifying products, not catching problems.
Of course they do, they're _certification_ companies, not tech support for security problems. Their job is not to catch problems in your software for you. It is to tell if a product is "secure" or not, according to tests. Which bring us to the point
1) You can't predict the future. Tests run today can't reproduce new problems that will be discovered next year. So this "security certification" is short-termed at least.
2) There is a bias, both in the test suite used and the conception they have of "security". They're human beings too, and to them "good enough" can mean a whole less (or more) than to you.
So what is the problem ? The problem is that apps that pass their tests is instantly classified as "secure". So we have to
- Expand the concept of "security" to give it a little more subjectify ("secure", according to company X, not just "secure, period).
- Use peer-to-peer review, which has proven good at detecting security flaws, and is quite inexpensive for free software projects.
Karma cannot be described by words alone.
Having been through a FIPS requirements meeting I generally agree with Schneir and Kocher: it can easily become a marketing tool if not taken seriously. While FIPS requires, say, certain crypto algorithms (DES, DH, DSA, etc) the physical boundary around the crypto hardware is pretty vague for level 1. Plus, as they mention in the article, you don't really know what method they use to test your product. Is it a monkey with a computer, a script, a Ph.D. mathematician, etc.
This removes the conflict of interest, and in fact reverses it: the certifying authority *wants* to find problems so they can bill more hours, and the developers but their butts to keep the cost of the certification down.
Jeff
To be useful for testing new crypto software, people need to be able to analyze source and algorithms and develop new attacks. Script kiddies, by definition, only carry out attacks built by others -- something an automated system can do just fine. Finding new vulnerabilities rather than working from a cookbook -- that doesn't take a script kiddie, it takes an expert (or five).
That's why analyzing crypto software is so friggin' expensive -- to do it right takes someone who knows a great deal about not only programming and info security but mathematics as well, and who has actual experience in the field. There are only so many people who can do it right (and I'm most certainly not among them); trying to get the job done properly using the average software engineer with 5 or 10 years of general (non-security-specific) experience won't work, much less a script kiddie of any variety.
And next you'll tell me that the "Nintendo Seal of Quality" was just a way to discriminate against third party game cartridges.
[PowerPoint] is a tool for capitalist presentation
talented script kiddies
Whoa, there's a phrase you don't see too often.
Wouldn't they be talented hackers/crackers, if they actually know their stuff?
The problem is that these certifications are often the only measure that PHB's use to decide if a product is 'safe' or not, and in this case, the certification is meaningless.
I can't count the number of times that a consultant has quizzed me about firewalls - when they're pushing a certain product (usually because they get a kickback) "This is better because it's certified!"
The problem is that (on off-the-shelf products) the certification only applies to the default configuration - and if you change it (which is pretty much every time - each site has different needs) the device needs to be re-certified... The consultants never mention that part to the client.
The best way to know if a site is secure is to have an independant security audit done by someone qualified (and I don't mean a 'general auditor' - a company that specializes in perimiter security.)
Now, before we all laugh and say "doesn't it show that the certifications are stupid?" consider this.. maybe the certification system does work, and all those other certified products are equally flaky. I've got a list of some TCSEC-certified systems here and frankly it's a pretty unappealing set of OSes. If there were as many Unicos systems (rated B1) out there as there were Windows, I betcha they'd find holes in it soon enough. The fundamental problem with any popular OS is that there will be thousands of hackers and wannabees probing away at it. I don't think there are many people reverse engineering CA-ACF2 MVS in their bedrooms.
I think the motto should be: "Security Through Obscurity" - perhaps all those horrid proprietry OSes did have a point after all.
Never email donotemail@WeAreSpammers.com
I agree that a certification isn't worth much if you don't know what it means. I think that's a big advantage of the Common Criteria (CC). While it's a lot to digest, there are guides out there to help you through it.
I work for the Canadian Common Criteria Scheme and it's my job to ensure that the Canadian labs follow the CC correctly and consistently in their evaluations. I found the article invaluable and disturbing (especially the Bruce Schneier quote), since we're obviously looking for ways to promote the CC, and the article highlights the concerns we need to address.
The same applies to those practices. In and of themselves, they do not guarantee that no incident will take place. But they'll hopefully minimize the impact and frequency of those incidents. The fact that the NSA or some other entity may be able to get past your security doesn't invalidate that security entirely; depending on the environment, it may be good enough.
Information security is really all about risk management. At the end of the day, are we managing our security to the point where the risk is less than the value of the information itself? Balance business need (or whatever needs you have, if you're not a business) against the cost of extra measures. When additional measures are too expensive for the value of what you're protecting, you're secure -- at least secure enough, anyway. If everyone followed security best practices, we'd have a lot less problems than we do.
"You can never have too many elephants on your team."
At http://www.oracle.com/ip/deploy/database/oracle9i/ you will see:
Oracle now holds 15 security evaluations. DB2 has none. SQL Server has only one.
If it was easy to "buy" these certifications, I'm sure that Microsoft SQL server would have more than just one by now. (Granted, Oracle also has a bit of cash to throw around too).
Slashdot: come for the pedantry, stay for the condescension.
From this article I get the impression that any Tom, Dick, or Harry can go out, 'perform testing' and give away FIPS certs for money.
This is not the case. FIPS 140-1/140-2 test labs must be approved by NIST through a formal accreditation program.
I was involved of getting our software package FIPS 140 certified, which is the major crypto security certification. I think there's some validity to the point that the certification house (which is sort of a gatekeeper to the actual certification) has something of a conflict of interest. We are paying them for the certification, while they are the ones who check the adequacy of our security measures. FIPS is supposed to check on their work, but that was largely a rubber stamp.
Nevertheless the certification house did do a thorough check on us and did recommend a number of changes to our software. We didn't think any of them truly added security, but at least this way it was obvious that the cert company was doing their job.
The big problem is that we got that version of the software certified, taking about eight months and several employees' time. Now a few months later we come out with a new release! We can't get re-certified every time, even though they have a shortcut for recertifications. Keeping up with the short software release cycle would be way too expensive.
So we still have FIPS 140 certification listed as a feature of our product, but if a customer really wants that specific version, we have to sell him old software. As it turns out, no one does. All they really need is to be able to check the box that says we are certified, and then they're perfectly happy to take the latest software. The mere fact that we spent the time, effort and money to be certified is what really counts.