Justifying the Common Criteria Security Evaluation

lewko writes "Microsoft has just received a Common Criteria certification for Windows 2000 at Evaluation Assurance Level (EAL) 4. Security experts have been saying for years that the the security of the Windows family of products is hopelessly inadequate. Now there is a rigorous government certification confirming this. What does it all mean? This paper suggests that Microsoft spent millions of dollars producing documentation that shows that Windows 2000 meets an inadequate set of requirements, and that you can have reasonably strong confidence that this is the case. Microsoft bashing aside, the process in evaluating a security product is relevant to anyone considering the deployment of technology into their environment." The EROS operating systems he mentions looks interesting - of course, it also looked interesting three years ago.

Comments

What's secure?

[Moridineas]

Well how much of what is secure? It seems to me that MOST of the security bugs one associates with Microsoft are problems with two programs in particular--IIS and Outlook (Express version only).

What did Linux get?

[mferrare]

I know some commercial Unixes are certified to C2 if you have it configured right. What about the Linuxes?

Re:What did Linux get?

[tuxlove]

Having helped develop C2 Unix OSes, I can tell you that Linux does not come close. There may be patches for all I know, but for sure stock Linux doesn't cut it. It's not a matter of Linux being buggy or broken; it's just not built to be that secure. I don't recall all of the criteria, but they are quite intrusive and the vast majority of Linux users would find them more than burdensome.

One example that immediately comes to mind is that "ps" listings can't show other users' processes. Many of the C2 requirements are kind of like that.

Re:What did Linux get?

[AftanGustur]

I know some commercial Unixes are certified to C2 if you have it configured right. What about the Linuxes?

Glad you asked. Some people might look at the fact that Linux doesn't have a XYZ 'certification' as a indication of that it is not secure enough to get it.

In reality, such certifications cost a lot of money and small companies like RedHat simply can't affort it (They don't make enough money of release X.Y during it's market-life, to justify such a operation)

What is interesting about this new Windows 2000 certification is that it's for a system that operates in a "safe" environment (i.e. not on the Internet) and Microsoft specifically asked, and paid, for grading at this level.

Now, you can interpret that as you want, but most of us are probably understanding it as "This is how secure Microsoft thinks Windows 2000 actually is". (Such gradings take a long time (few years) and I doubt that Microsoft will have another go at a higher grading before the EOPL (end-of-product-life) of Windows 2000.

Re:What did Linux get?

[debrain]

ISO/IEC 15408 (supercedes the rainbow series, to which CCITA belongs): ACL's, sudo (or equiv.), auditing, non-root'ing (ie. linux single) boot params, non transferrable FS. All are available for Linux, but more likely you will use or want to use a BSD.

I don't consider ISO/IEC15408 machines a burden, especially in lieu of the alternatives; most user-level programs may never tell they are there. As Linux is source, it is trivial (well, insofar as kernel hacking is trivial; maybe 'possible' is a better word) to acquire the necessary options for ISO/IEC15408. It seems easier to do this with BSD's because they tend to be simpler in design.

Re:What did Linux get?

[Zeinfeld]

Glad you asked. Some people might look at the fact that Linux doesn't have a XYZ 'certification' as a indication of that it is not secure enough to get it.

In reality, such certifications cost a lot of money and small companies like RedHat simply can't affort it (They don't make enough money of release X.Y during it's market-life, to justify such a operation)

No, Linux would fail evaluation because it does not meet many of the important security requirements. In particular there is no system security guide that describes how to securely configure the O/S in a single place.

Documentation is a large part of the C2 criteria. Linux simply fails that test. You cannot get certification for a third party guide for good reason, the document has not been reviewed by the engineers who wrote the code.

It is interesting to note how the Fox News style bias of slashdot on the security topic gets more hysterical by the month. Could it be because analyst firms like Aberdeen are predicting that Linux will become the poster chid for security, and no they don't think it is more secure.

So Microsoft get a security evaluation, the slashdot response is to publish the story three times to date, each time claiming that it is further proof that Microsoft's products are insecure. At what point do people ask whether the Slashdot editorial style has more to do with the commercial interests of their employer than an interest in honest journalism?

Well

[BrianGa]

This kind of certification is a great thing for people running Win2K. But I have to wonder if Microsoft's upgrade cycle will cause those people to lose official support for Win2K unless they upgrade to XP or whatever's next very soon now? A lot of enterprises do a lot of time-consuming testing before they rollout something like Win2K, which is probably the first reasonable OS from MS. It'd be a real shame if all that testing and certification gets thrown out the window because MS doesn't feel its customers are buying upgraded products fast enough.

one basic reason why windows security sucks

[Indy1]

Before you mod me down, I am a network admin that works with both windows 2000 and linux on a daily basis. I am also a certified MCSA (though we all know what we think of certs :) ). Anyways, my #1 reason why i think windows security SUCKS, is that the damn OS has no real firewall built into it! I mean, come on, with win2k you gotta either buy a hardware firewall (cisco pix, etc), or throw a unix box in front of it. And yes, i know XP does have a basic firewall built in, but do any of you want to run a server on XP ? People always bitch at MS for bundling software into their OS, but there's no excuse to not include reasonable packet filter ability in the OS. Thats why I believe the only time you EVER put a MS box on the net is if its behind a NAT or something else that totally hides the box from outsiders.

Re:one basic reason why windows security sucks

[sheldon]

"I am also a certified MCSA"

and then later on...

"People always bitch at MS for bundling software into their OS, but there's no excuse to not include reasonable packet filter ability in the OS."

Well you've certainly proved one thing. People with certifications can often oversell themselves as experts when they really know very little about the products.

Psst... I share the bounty of a simple google search.

Forgive my ignorance...

[MoThugz]

but who the hell are the CommonCriteria folks, and why must I give a shit what they think of whatever OS?

The above is an honest question, if you can't elaborate clearly, please don't even bother to reply.

Thank you.

Re:There are real, secure, systems out there.

[Student_Tech]

It is interesting to read about the concept of rings, with the main kernel sitting in the innermost ring. I know that some of the computers my dad worked on out at Hanford were Primes that had some concepts of rings, with the innermost being the highest up on the privilege levels. Not exactly the same I don't think, but this was many years ago.
(Hey just went looking, looks as if Primos Revision 21.0.1DODC2A got to the C2 level, so maybe this is more similar then I think)

Re:EROS: The Extremely Reliable Operating System

[pVoid]

A properly configured Windows Box can be just as secure as any OS, you just have to know the system

Thank you for saying that out... there is nothing more valuable than a sysadmin who knows his platform.

I've been hearing a lot of moft-is-not-secure 'proofs' lately... I'm just wondering: has anyone actually proven that the OS is structuraly (ie by design) flawed?

A structural flaw for example would be that files have ACLs, but pipes don't. Or something of the sort... *not* that the default out of the box configuration leaves a NULL ACL on the \system32\cmd.exe (that is not a structural problem, it's configuration).

So long as someone doesn't show real facts when they claim to 'proove' something, it's FUD pure and simple AFAIConcerned.

Re:Theo de Raadt

[alizard]

The only way Microsoft(C) can get a secure OS is to throw ton of money at Theo de Raadt or somebody with his mind set to run a team to continually check Windows code for security problems.

You mean have MS pay Theo and everyone connected with the OpenBSD project enouh to persuade them that taking it proprietary and rebranding it Windows XX is A GOOD IDEA, right? Continuously checking Windows OS and applications for security fuckups is too big a job for one person, and probably too big a job for 1,000 persons.

Would the OpenBSD team sell out for $10 billion and the right to oversee future development?

Note that this would actually be an intelligent and cost-effective thing for MS to do, even if various code libraries have to be rewritten to avoid the use of GNU code of any sort, so we can take for granted that they'll never think of it for themselves.

While this is a lot more than MS paid for the rights to what later became MSDOS ($30K, IIRC), times have changed.

While this breaks compatibility with all MS applications, does anyone actually think anything less has the remotest chance of doing the job? Assuming the job is building a reasonably secure OS that can be made to work with a wide range of applications.

Re:EROS: The Extremely Reliable Operating System

[vadim_t]

Users? You need no users for that.

This means that any program can screw my registry enough to leave the system unbootable. What's the point in running as normal user, then? Just try to rm -rf /etc on Linux. I'm pretty sure that unless you're root it'll still work fine afterwards. And that's how it should be.

On Linux, if I want to try a suspicious program I can create a new user account and try it there. If I want to be more paranoid I can chroot it and use strace to find what exactly it's doing.

Now, if in Win2K it's possible to break the whole system as a normal user, where's all that security it's supposed to have?

Also, what registry tree? I've seen no detailed help files explaining every key of the Windows registry, what it's used for, and what would happen if it had too restrictive permissions. If those permissions are so badly set from the beginning it makes me think the reason is that many programs will break when they're unable to write to some places. If changing those ACLs would give me better security at the cost of breaking half of my programs, thanks, I don't want it. Linux works much better.

Common Criteria

[drsolly]

I bumped into this several years ago, in the antivirus field. "Get the product certified", said the marketing department. "Some big corporates want to see an official certification" said our sales people.

So I looked into it. At the time, it was called "Itsec", now it's "Common Criteria". It was run, in those days, by the electro-spooks, based in Cheltenham.

When I found what it was, I was absolutely ROFL.

I, the vendor, was expected to state the functionality of the product, what it was supposed to do, security-wise. They call this the TOE, "Target of Evaluation"

They, the evaluators, would check that it met that functionality, and give me a certificate if it did.

So far, so good. But what's the right functionality? In my case, what functionality should an antivirus have (rhetorical question, please don't tell me, except it isn't as simple as you might think).

So, I said to the people who ran the scheme, Suppose I define my functionality as "Comes in a blue box". Could I get an Itsec certification for that? The answer boiled down to "Yes, but that isn't a security issue". "Yes it is," I said.

Um. Who defines what is a security issue and what isn't? I was saying that the lack of a blue box, was a security issue. How do you say it isn't? Anyway, that's my TOE, please certify it. Well, it never got that far, that was just my way of telling them that their scheme was a joke.

So I went to a pal of mine who ran the security department at a university, suggested that he set up a certification scheme, and got the product certified under that instead. That made our marketing people happy, also our sales people. Customers had a certification to pin on the wall, everything was tickety-boo.

Except the government people, who knew they were being made monkeys out of, because I threw that "Comes in a blue box" thing at them at every conference and seminar I went to, and I heard that it started to seriously embarrass them, because people started asking questions about the value of their certifications. There's more in that thread - things did start to change, but the change didn't happen in the end.

Now, I'm not suggesting that the Microsoft certification says "Comes in a blue box." But until you've read the TOE, you don't actually know what security functions have been certified.

Re:EROS: The Extremely Reliable Operating System

[ergo98]

It is *very* flawed. This is how you can destroy a Win2K system as a normal user

Despite your probable pleas to the contrary, you were not a regular user when you carried that out. Windows has ACLs on virtually everything in the OS (contrary to Linux, for example, with its incredible large granularity security), and the registry is no exception. The HKLM registry branch has only READ access for anyone but System and Administrators (in some cases also Power Users, which much like Administrators is not an account that you should regularly run under). The registry applications abide by these permissions quite simply because they can't fail not to. I see two possible scenarios here, one that you were in an account as PowerUser or Administrator, or two that there is a complex fault that somehow bypassed the ACLs. I suspect the former as being dramatically more likely.

Having said, you weren't actually trying to do that in a serious way, were you? (copying the tree from 98 to 2K) As a sidenote, virtually all Windows variants keep one or more backup to the registry tree, and choosing "last known good configuratin" would have fixed it for you immediately.