Slashdot Mirror

My theory is that avast didn't ask to have their product evaluated so no government asked for their code to evaluate. To be able to sell security products to a lot of governments you need to be evaluated. Common criteria is an international group that standardizes and recognizes the evaluations across its members. Being CC evaluated puts you on the shopping list for a reasonably large government market.
For a list of products that have had at least one government(or their representatives) crawl through the code https://www.commoncriteriaport...

Computers sometimes fail. So do humans. The best way to not be at the mercy of either is to have both. There's at least one incident on record where malfunctioning sensors told a plane computer that it was 4000 feet higher than it actually was, and it would've happily crashed into the ground during descent if the pilot hadn't looked out the window to say "wtf, that's the ground right there".

At HAL 2001, yes that was 14 years ago, there was a speech with the title "why my space ship will not run Linux" and that's as true today as it was back then: Our current software, from firmware and operating system to applications, is total crap, incredibly shoddy, and half of it is being held together by spit and duct tape. Fact is that while we make progress (and not a little, actually it's quite amazing), we still don't know how to write really good software. We know a bit about how to teach humans to write pretty good software, even though most companies use 10% of that knowledge in real production (mostly because next-quarter focussed managers don't understand the incredibly good ROI on high software quality).
But a lot of that knowledge is about software development processes.

But do we know how to make a non-trivial computer program that is guaranteed to behave correctly? How much software with an EAL5, EAL6 or EAL7 certification do you know? Wait, you can check here. Not very many.

Yes, a static baseline is great for certification programs such as EAL and FAA approval, but it's not the only sort of "stable" that you want. Data centres want a "carrier-grade" OS (which means five nines reliability). They don't necessarily care if they have to patch, since you can now hot-patch the kernel without taking it down, but they absolutely do not want the software to show any unreliability whatsoever. They'd likely get upset at having to patch more than once a year, since in-situ patching isn't always safe, but if you're limited to a few minutes downtime a year on a server as an absolute maximum (this is ignoring failover, etc, that's a whole different issue than a specific physical or virtual server instance being five nines) then I could see it being tolerated a whole lot more than a blind kernel upgrade at year's end.

(This assumes that the hot upgrades can be made fault-tolerant enough that a brown-paper-bag release - you know they're going to happen on any tree eventually - can be backed out without violating five nines.)

These are what systems are required to do in the way of security measures, as defined by the Federal Information Processing Standards, the Orange Book and the Common Criteria.

A lot of the documentation can be found at the Information Assurance Support Environment website, Policy and Guidance

To summarize, information that is labelled "Secret" can only be stored on a machine that - in the Orange Book system - is classed as B3 or better. The use of security labeling and a mox of host-level and network-level mandatory access controls is supposed to ensure that this is actually mandated at the OS level on each machine and between machines. B3 is equal to the more modern Commmon Criteria EAL4.

(It is impossible, in theory, to transfer information that is classified at one level into a lower classification, on the same machine or by going through a series of machines. To be able to do so is a violation.)

To be given an EAL4 rating, that precise combination of hardware and software MUST be tested by an approved laboratory and shown to meet all of the criteria.

Further, as noted on the FIPS website: "With the passage of the Federal Information Security Management Act of 2002, there is no longer a statutory provision to allow for agencies to waive mandatory Federal Information Processing Standards (FIPS)."

• Minimum Security Requirements for Federal Information and Information Systems (FIPS 200)
• Standards for Security Categorization of Federal Information and Information Systems (FIPS 199)
• Standard Security Label for Information Transfer (FIPS 188)

Mandated Criteria, Rainbow Series and Related

• Computer Security Requirements (CSC-STD-003-85)
• Security Requirements for Automated Information Systems (AISs)
• A Guide to Understanding Configuration Management in Trusted Systems (Orange Book, Rainbow Series)

Mandated Criteria, Common Criteria

• Common Criteria for Information Technology Seciryt Evaluation, Part 1
• Common Criteria for Information Technology Seciryt Evaluation, Part 2
• Common Criteria for Information Technology Seciryt Evaluation, Part 3
• Comon Methodology for Information Technology Security Evaluation

These are NOT optional. These are Federally-mandated requirements. If Manning's computer did not meet these standards, it was NOT authorized to be on the network and the machines that transferred classified information to it were NOT authorized to do so.

These are what systems are required to do in the way of security measures, as defined by the Federal Information Processing Standards, the Orange Book and the Common Criteria.

A lot of the documentation can be found at the Information Assurance Support Environment website, Policy and Guidance

To summarize, information that is labelled "Secret" can only be stored on a machine that - in the Orange Book system - is classed as B3 or better. The use of security labeling and a mox of host-level and network-level mandatory access controls is supposed to ensure that this is actually mandated at the OS level on each machine and between machines. B3 is equal to the more modern Commmon Criteria EAL4.

(It is impossible, in theory, to transfer information that is classified at one level into a lower classification, on the same machine or by going through a series of machines. To be able to do so is a violation.)

To be given an EAL4 rating, that precise combination of hardware and software MUST be tested by an approved laboratory and shown to meet all of the criteria.

Further, as noted on the FIPS website: "With the passage of the Federal Information Security Management Act of 2002, there is no longer a statutory provision to allow for agencies to waive mandatory Federal Information Processing Standards (FIPS)."

• Minimum Security Requirements for Federal Information and Information Systems (FIPS 200)
• Standards for Security Categorization of Federal Information and Information Systems (FIPS 199)
• Standard Security Label for Information Transfer (FIPS 188)

Mandated Criteria, Rainbow Series and Related

• Computer Security Requirements (CSC-STD-003-85)
• Security Requirements for Automated Information Systems (AISs)
• A Guide to Understanding Configuration Management in Trusted Systems (Orange Book, Rainbow Series)

Mandated Criteria, Common Criteria

• Common Criteria for Information Technology Seciryt Evaluation, Part 1
• Common Criteria for Information Technology Seciryt Evaluation, Part 2
• Common Criteria for Information Technology Seciryt Evaluation, Part 3
• Comon Methodology for Information Technology Security Evaluation

These are NOT optional. These are Federally-mandated requirements. If Manning's computer did not meet these standards, it was NOT authorized to be on the network and the machines that transferred classified information to it were NOT authorized to do so.

These are what systems are required to do in the way of security measures, as defined by the Federal Information Processing Standards, the Orange Book and the Common Criteria.

A lot of the documentation can be found at the Information Assurance Support Environment website, Policy and Guidance

To summarize, information that is labelled "Secret" can only be stored on a machine that - in the Orange Book system - is classed as B3 or better. The use of security labeling and a mox of host-level and network-level mandatory access controls is supposed to ensure that this is actually mandated at the OS level on each machine and between machines. B3 is equal to the more modern Commmon Criteria EAL4.

(It is impossible, in theory, to transfer information that is classified at one level into a lower classification, on the same machine or by going through a series of machines. To be able to do so is a violation.)

To be given an EAL4 rating, that precise combination of hardware and software MUST be tested by an approved laboratory and shown to meet all of the criteria.

Further, as noted on the FIPS website: "With the passage of the Federal Information Security Management Act of 2002, there is no longer a statutory provision to allow for agencies to waive mandatory Federal Information Processing Standards (FIPS)."

• Minimum Security Requirements for Federal Information and Information Systems (FIPS 200)
• Standards for Security Categorization of Federal Information and Information Systems (FIPS 199)
• Standard Security Label for Information Transfer (FIPS 188)

Mandated Criteria, Rainbow Series and Related

• Computer Security Requirements (CSC-STD-003-85)
• Security Requirements for Automated Information Systems (AISs)
• A Guide to Understanding Configuration Management in Trusted Systems (Orange Book, Rainbow Series)

Mandated Criteria, Common Criteria

• Common Criteria for Information Technology Seciryt Evaluation, Part 1
• Common Criteria for Information Technology Seciryt Evaluation, Part 2
• Common Criteria for Information Technology Seciryt Evaluation, Part 3
• Comon Methodology for Information Technology Security Evaluation

These are NOT optional. These are Federally-mandated requirements. If Manning's computer did not meet these standards, it was NOT authorized to be on the network and the machines that transferred classified information to it were NOT authorized to do so.

These are what systems are required to do in the way of security measures, as defined by the Federal Information Processing Standards, the Orange Book and the Common Criteria.

A lot of the documentation can be found at the Information Assurance Support Environment website, Policy and Guidance

To summarize, information that is labelled "Secret" can only be stored on a machine that - in the Orange Book system - is classed as B3 or better. The use of security labeling and a mox of host-level and network-level mandatory access controls is supposed to ensure that this is actually mandated at the OS level on each machine and between machines. B3 is equal to the more modern Commmon Criteria EAL4.

(It is impossible, in theory, to transfer information that is classified at one level into a lower classification, on the same machine or by going through a series of machines. To be able to do so is a violation.)

To be given an EAL4 rating, that precise combination of hardware and software MUST be tested by an approved laboratory and shown to meet all of the criteria.

Further, as noted on the FIPS website: "With the passage of the Federal Information Security Management Act of 2002, there is no longer a statutory provision to allow for agencies to waive mandatory Federal Information Processing Standards (FIPS)."

• Minimum Security Requirements for Federal Information and Information Systems (FIPS 200)
• Standards for Security Categorization of Federal Information and Information Systems (FIPS 199)
• Standard Security Label for Information Transfer (FIPS 188)

Mandated Criteria, Rainbow Series and Related

• Computer Security Requirements (CSC-STD-003-85)
• Security Requirements for Automated Information Systems (AISs)
• A Guide to Understanding Configuration Management in Trusted Systems (Orange Book, Rainbow Series)

Mandated Criteria, Common Criteria

• Common Criteria for Information Technology Seciryt Evaluation, Part 1
• Common Criteria for Information Technology Seciryt Evaluation, Part 2
• Common Criteria for Information Technology Seciryt Evaluation, Part 3
• Comon Methodology for Information Technology Security Evaluation

These are NOT optional. These are Federally-mandated requirements. If Manning's computer did not meet these standards, it was NOT authorized to be on the network and the machines that transferred classified information to it were NOT authorized to do so.

The thing I keep seeing is lazy DISA auditors that see the STIG's as black and white. Most of the testers I've run into aren't technical people. They run the automated SRR scripts and ding you for having your kernel version out of spec. If I were to sit them down and ask why a particular control was an open finding they'd tell me "Because the STIG said so" without digging deeper as to why.

The most recent test I was on, the testing team hit the sys admins for an out of date Kernel on a VMWare ESX box. VMWare uses a highly customized version of RHEL. Installing the most recent Kernel would turn the box into a paperweight. The best advice I can give you is to first check with the tester to find out exactly what the vulnerability is and what their recommended fix action is. Depending on your tester you may be wasting your time. I've see far too many tester leave comments like "Not up to STIG compliance". Check with your vendor to see if they have issued a patch to address that vulnerability. Once you have that information you can place your comments into a POA&M and go back to your DAA and explain why a given open finding isn't really a finding and/or won't be fixed. You can also look into mitigation factors to see if you can reduce the severity. Many controls will state "If you're doing X, Y and Z this finding may be reduced from a CAT I to a CAT II".

Good luck with your C&A and be glad you're not on the documentation side of things :^)

Evaluation by Common Criteria Portal:

Microsoft Windows Vista and Windows Server 2008. Evaluation Level: EAL1
Miracle Linux:EAL1
Red Hat Enterprise Linux Version 5.1:EAL4+
Microsoft Windows Server 2003 SP2; Windows XP Professional SP2 and x64 SP2; Windows XP Embedded SP2 EAL4+

Microsoft Windows Server 2003 and Microsoft Windows XP EAL4+

http://www.commoncriteriaportal.org/products_OS.html#OS

This at least shows that Vista is total swiss cheese and that much-patched 2003/XP is in the same ballpark as some linux distributions.

That must be why Windows Vista has an astronomically awesome EAL1 rating by NIAP labs (link to PDF), why Windows Vista SP2 is trusted by the US government to divide classified networks (oh, wait, did I say Windows? I meant Solaris, SELinux, or HP/UX, mybad), or why my Vista PC got infected with Conficker while my Linux box hasn't had such problems (and it's in my DMZ, while my Vista box ain't!).

Go Microsoft Go.

That must be why Windows Vista has an astronomically awesome EAL1 rating by NIAP labs (link to PDF), why Windows Vista SP2 is trusted by the US government to divide classified networks (oh, wait, did I say Windows? I meant Solaris, SELinux, or HP/UX, mybad), or why my Vista PC got infected with Conficker while my Linux box hasn't had such problems (and it's in my DMZ, while my Vista box ain't!).

Go Microsoft Go.

EAL-1 what? http://www.commoncriteriaportal.org/products_OS.html#OS

Some companies do use it for marketing. Others use it secondarily for marketing, but primarily to garner or maintain eligibility for certain contracts. EAL certification is required to get into certain government roles, for example. Ongoing re-certification is required to remain in some of them. The criteria and results are available from the Common Criteria website. For example, the evaluation covering Windows XP and Server 2003 details the OS variants, hardware on which it was tested, and drivers and patches that were present during testing.

Really, I was just being snarky: if you or I test drive a few different operating systems with our (or our organization's) needs in mind and we call it formal, we get laughed at. If this guy does so, he gets to not only call it formal, but also delineate the temporal boundaries of the formal period of his evaluation. It's meaningless. He took a test drive. Call it that.

In a world where I was potentially being serious, I might have responded to your arguments thusly:

since you accuse me of question-begging, i will answer the question you allege i begged, namely to outline the varyingly "formal" evaluation pathways. The currently accepted norm is that two markets for software products require evaluation: safety-critical and security-critical. Some few systems have requirements for both, but rarely is this acknowledged. The reason for this is the evaluation standards for each community are quite burdensome, but have relatively little overlap, even though one might think they would.

In the US especially, the safety critical community is divided further, into avionics and medical; evaluations are overseen/conducted by the FAA and FDA respectively. The two primary standards the FAA evaluate under are DO-178B and ARINC-653. Each has several levels of scrutiny depending on the potential consequences of failure of the software at hand. None of them are formal in the sense that properties about the code are proven mathematically. They are instead formal in that a list of functional requirements is provided, as is a traceability matrix that links the specification to the code that implements it and vice versa. Typically the higher levels of evaluation mandate things like an absence of extraneous code resident on the system. Current safety evaluations are not modular, and have to be fully reiterated even on the smallest change to the software. If you were to buy an OS that has been in a product that has been evaluated, you would also want to buy their evaluation evidence or else you'd have to reproduce it.

Security critical software is evaluated, in the countries that are signatories to the Common Criteria Treaty (forget its actual name), under the common criteria. Again, it has a variety of levels of intensity of evaluation, but more tricky is the fact that there is another variable: the anticipated use and threat environment, known as a protection profile. The Common Criteria website explains it far better than I'm capable of: http://www.commoncriteriaportal.org/

This site includes a list of operating systems that have gone through evaluation. Most of them are evaluated to levels 3-4 on a scale of 7, which seems fairly good until you examine their protection profiles. Most of those assume no malicious users, and a variety of other restrictions that preclude consideration of threats common in most deployment scenarios. The reason for this is that vendors want to garner marketing cachet by being able to claim a high evaluated assurance (EAL) level, assuming the multidimensional system will confuse prospective buyers. This happens on both sides of the MS/everyone else isle.

There is also a page on that site for products that have gone through evaluation, and in the US flavored site there is a list of products under evaluation: http://www.niap-ccevs.org/cc-scheme/in_evaluation. cfm.

So, the evidence from an evaluation is indeed closely held by the companies that have products evaluated, but the idea in having a "neutral" third party evaluate under a more broadly common set of criteria was to shift away from groups closely holding net results, while allowing those results to still be meaningful, especially for comparing different companies' products. You see, there is no competitive advantage in being evaluated and not telling anyone your score.

The Common Criteria is a joint project between several governments including Australia: http://www.commoncriteriaportal.org/

If you need to supply computer equipment to a government agency then you better start reading. If not, then don't bother.

There seems to be a fairly significant amount of ignorance on this topic.
Some information is available, but it is as complex as the systems which need securing. The basic idea is to give a number that indicates the quality with which security is maintained. ALC_FLR.3 is broken down to mean:

1. ALC = class of life cycle support
2. FLR = family of flaw remediation
3. 3 = level of ranking

In particular, at ALC_FLR.3 it means that you have procedures for reporintg flaws, you fix the flaws, and you tell users how to get the fixes. All of that must be verifiable and documented. It is the highest ranking in that category. There are many other categories, however.

Furthermore, there are additional protection profiles that may be invovled. Because of the difficulty of getting certifications, they are typically provided with a specific hardware baseline and a specific configuration. So, Sun's Trusted Solaris 8 was EAL4, but also has LSPP on top of it.

Well, what does all that mean? It means that evaluated products may be configured securely, but not that they are secure innately. Effort is required. In practice, having an EAL rating is an initial requirement for an OS. In some cases there is an minimum rating for using the software. Sometimes it extends beyond the OS. All of that depends upon the contract. EAL4 is generally considered the minimum rating to be viable in a "low risk" environment.

Microsoft is only certified CAPP/eal4+. That is not LSPP/RBAC which is much harder and more secure.

• CAPP: Controlled Access Protection Profile
• RBAC: Role-Based Access Control
• LBAC: Lattice-Based Access Control

I doubt this.

There's no way they could've received EAL4 (see the product list) without good documentation. The CC have a very strong focus on documentation and EAL4 is not something you'd get with shoddy and incomplete docs.

Now I haven't studied the evaluation target, so I can't say for sure just which APIs it includes and which ones not. Also note that the certifications are for 2000, 20003 Server and XP only and your experience predates those, so yes, M$ has probably cleaned up shop.

For those who don't have the foggiest... More info on Common Criteria Certification can be found Here

Sigh!

The link you refer to points to material that is up to two decades old. The assurance levels you refer to (A, B, and C) are from the Orange Book, the seminal work of the Rainbow Series of security development manuals produced for the U.S. DoD.

The Rainbow Series was superceded in 1996 by the Common Criteria, an international agreement about security functional requirements, assurance requirements, and the processes needed to evaluate the security characteristics of IT products. Products that have met the requirements and undergone the process are listed in an Evaluated Products List. Among operating systems that have met the Common Criteria requirements are Mac OS X, Red Hat Enterprise Linux AS/WS 3, Solaris 9, SuSE Linux Enteprise Server V8, and Windows 2000 Server. All of these must be run on specific hardware configurations and with specific software configurations to retain their certified status in an operational environment. A recent project I was working on needed an HTML-based interface - imagine creating that on a Linux box that could not run X or even activate the frame buffer!

Secure systems are not just platforms that resist the latest script kiddie 'sploit. A system includes people, processes, hardware, software, development methodologies, and the operational environment. This is what makes a secure, assured SYSTEM, not just an expensive doorstop.

Links of (possible) interest:

Orange Book
http://csrc.ncsl.nist.gov/secpubs/rainbow/std001.t xt

Rainbow Series
http://csrc.nist.gov/secpubs/rainbow/

Common Criteria
http://www.commoncriteriaportal.org/

U.S. "Scheme"
http://niap.nist.gov/cc-scheme/

Evaluated Products List (EPL)
http://niap.nist.gov/cc-scheme/vpl/vpl_type.html#o peratingsystem

I'm so tired of this argument "Our software is more secure than their software". It's ridiculous. What they're really saying is "Our programmers and development processes are better than your programmers and processes." These security debates, whitepapers, and arguments are always subjective, never solve anything, and only prove that someone has time to waste.

Any given OS, in the hands of an expert, is just as stable or secure as the next. There's just no way to effectively prove otherwise. The test domain to definitively prove which OS is truly the most secure is incredibly huge. As long as human beings code it, it's insecure. There is no version of Unix or Linux that has a higher Evaluation Assurance Level than Windows 2000. That doesn't necessarily mean that any novice could actually secure it either.

Reality is that Windows has a huge number of desktop installations and it's used by a large number of people that can't even open up Notepad or a command prompt if you asked them to. Those same people couldn't even install Linux so it's not reasonable to even suggest. So, how are they supposed to have any idea about security? Most of them can barely get online. It's no fluke that AOL and Windows are as popular as they are - they're easy to use and they have a small learning curve.

Furthermore, Linux and Windows are so different that's almost ridiculous to even compare them. They solve different problems, they both have their strengths and weaknesses, and other than the fact that they're both operating systems they don't have much else in common. In many ways comparing those two systems is like comparing an F-16 to a Leer jet - they both fly; they both have wings; they both have cockpits, throttles, and tails; they're both airplanes but they don't look the same; they don't have the same internal components; they aren't operated the same; and they aren't made for the same purpose.

Security arguments are out of style. It's safe to say that no major software maker is intentionally designing insecure software. Move on. Innovate. Come up with something original.

Incorrect. Red Hat Enterprise Linux 3 was certified EAL2 in Feburary 2004. The certification was sponsored by Oracle. See: http://www.commoncriteriaportal.org/public/consume r/index.php?menu=4&orderindex=1&showcatagories=256

Nobody makes software to the A1 standard. At least, not that anyone is admitting.

GEMSOS is a general purpose A1 rated operating system kernel.

Since the author asked for constructive feedback, I sent him an email about the common criteria issue you were pointing out. His response basically says that he already explicitly mentioned that the orange book equivalency is _approximate_, and pointed me to the following pdf on the CC web site, which suggests similar equivalency. Bottom line: mappings can be used to explain the new ratings in light of the old, but are not strict. So their use is not totally out of whack, if you take it with a grain of salt.

CC Introduction

Everything changes when the attacker has physical access to your hardware (as others have pointed out).

The Common Criteria is an internationally-recognized standard, so the U.S. gov't would recognize the German EAL3 augmented evaluation of SuSE Linux Enterprise Server V8 that just finished up in January 2004.