Slashdot Mirror
Virtualization Decreases Security
ParaFan writes "In a fascinating story on KernelTrap, Theo de Raadt asserts that while virtualization can increase hardware utilization, it does not in any way improve security. In fact, he contends the exact opposite is true: 'You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes.' de Raadt argues that the lack of support for process isolation on x86 hardware combined with numerous bugs in the architecture are a formula for virtualization decreasing overall security, not increasing it."
No folly is more costly than the folly of intolerant idealism. - Winston Churchill
The Irish Potato Famine happened because Ireland was growing a small range of species of potato.
A virus hit the Potato and it spread so most of the potato's died thus causing the famine.
Other Areas had the same virus but it didn't cause a Famine because their stock was more diverse.
It wasn't because the other guys potatoes were immune to all virus or they were a heartier bread, but
because they had a wider diversity of product.
The same thing with Virtualization, each VM will not be completely secure and will have holes in it but
spreading will be reduced because only a smaller portion of application will use that OS to virtualize.
A Linux VM OS, a BSD VM OS, a Windows VM OS... Sure there will be security problems and patching and fixing
the problems in the Virtual OS will need to be resolved... But if there is an outbrake you will basicly loose your
VM application and perhaps some other ones that you may have running at the same time that uses the same OS. But now
If your OS Gets infected all your Apps are dead.
If something is so important that you feel the need to post it on the internet... It probably isn't that important.
You're just NOW realizing this???
Virtualization layers can be much smaller than operating systems. Hypervisors don't have to do as much as a monolithic kernel does, so they're less prone to security holes.
You've been smoking something really mind altering, and I think you should share it.
x86 virtualization is about basically placing another nearly full kernel, full of new bugs, on top of a nasty x86 architecture which barely has correct page protection. Then running your operating system on the other side of this brand new pile of shit.
You are absolutely deluded, if not stupid, if you think that a worldwide collection of software engineers who can't write operating systems or applications without security holes, can then turn around and suddenly write virtualization layers without security holes.
You've seen something on the shelf, and it has all sorts of pretty colours, and you've bought it.
That's all x86 virtualization is. It's highly probable that Theo is right. After reading the above post, it's highly probable he is a very abrasive and one sided individual. But this is a tech forum so I won't get into judging character.
However, his technical argument is
I'm going to point out some other things I know about coding. Although more lines of code usually means more bugs, this is not always the case. Correlation does not equal causation. It is correlated but only because the more lines of code, the more probability that more people contributed to the project which means it is highly probable one of them was a bad coder. Also, if you plan things out and follow a rigorous model, it is within your power to make very fully functional, very nice software.
My second point is a different way of looking at the problem. Let's take the naive approach of assuming a primary job of the operating system is to protect the user (and applications) from completely fouling things up in the hardware & memory realm. So it does an 'ok' job at this but, as Theo noted, some bugs still exist. Let's say it's something really bad like they don't stop programs from altering a very sensitive range of memory that is very vital to the correct execution of the operating system itself. Now, hypothetically, the virtualized layer on top of this would give coders a chance to catch this and correct it and protect the user from bringing down the operating system. In this way of looking at things you have two nets. Alone one lets many things pass through so you double it up and now you're catching more fish.
But my analogy is probably very flawed and I must confess I have coded neither of these pieces of software so I cannot confirm or deny this. I am quite shocked that Mr. de Raadt would react so abusively to a post where someone was merely saying that they 'appeared' to be receiving some amount of additional security from virtualization.
As for the very last comment Mr. de Raadt makes, I am confused. My employer uses virtualization on a mass scale to more effectively utilize hardware. I believe it has more uses than just bright shiny colors and wrapping--in fact I am interested in its potentials for hosting web OSs and other neat applications to users. It might not be the future like some people think it is but I think Mr. de Raadt was suffering a moment of frustration or dealing with irritable people when he authored this.
I do wish he were open to more ideas. The second you start to just outright dismiss all your options because they don't satisfy you on the surface you will find you are left with none and often miss the best.
My work here is dung.
Theo thinks so highly of himself, he is just wrong on this one. There is not one recorded/public example of someone breaking out of the isolation of a virtual environment! I dare someone to demonstrate otherwise, and I will eat my words.
This guy is just grabbing for attention. Anyone in IT already knew there will be security issues when introducing a Hypervisor, but it doesn't make your hosts/lpars any less or more secure. A good IT admin designs systems to be secure by identifying potential security risks and minimizes them and there is no reason to let this clown spread FUD that might inhibit a fast growing technology advancement.
I'll make a bold statement that hypervisors make systems MORE secure because they obscure the hardware layer and allow for cleaner more efficient drivers to be installed uniformly on your accessable hosts.
Thanks for the insider tip Theo, I just dumped all of my VMware stock.
Virtualization layers, and their cousins Separation Kernels are the darlings of the security crowd because they can be written in a relatively small number of SLOCs, which means there is a possibility to formally analyze them. Where Formal Analysis means proofs written in a well established mathematical notation, and machine checked. Green Hills Software has a separation kernel that should shortly be certified to a very high level (EAL 6 Augmented) CCEVS
A few years ago, it seemed email worms were constantly ravaging Outlook. That, I noticed. But that seems to have tapered off. Haven't noticed any panicked patching of zlib or ssh or sendmail lately. What is keeping people busy these days? Spyware-infested zombie boxes? Anything else?
Virtualization is no doubt a complex problem to get right, but it's only one problem. There is a relatively fixed set of hardware any virtualization system claims to support. A reasonably complete virtualization system can be frozen at some level of functionality. An operating system can not; it must, by nature, constantly evolve to new requirements. Hardware, in contrast, is relatively more stable.
Operating systems running on virtualized systems also have the advantages of operating systems running any fixed configuration. While not quite as consistent as a completely emulated environment, virtualization gets most of the benefits, under reasonable assumptions.
So, in short, virtualization has the same sort of benefits microkernels were supposed to provide, albeit with a much more heavyweight solution: smaller core that's easier to secure. Virtualization has been used in the mainframe community for years. Virtualization is an even stronger form of process isolation than what operating systems provide.
Virtualization is much more costly to run than a standard operating system process. This should be a clue that it probably provides stronger isolation guarantees, even if you don't buy the rest of the argument.
I think it's a specious argument, as usual, to claim that securing the virtualization layer is no harder or easier than securing an operating system. I think securing the virtualization layer is going to be much easier, because while the problem itself is complex, it's still less complex than a complete operating system is.
A better argument would have been to point out that guest operating systems running under virtualization are no less vulnerable to being compromised than those running on real hardware. But then that would point the finger at operating system vendors, not virtualization ones.
From TFA,
The topic is specifically about virtualization on the x86 platform.
Theo's side keeps asserting that "x86 virtualization isn't secure", but they seem to be perfectly comfortable at keeping the discussion at the level of a "I'm right, NO I'M RIGHT", without any corroborating statements (Hint: Theo's "I am familiar with x86 and its 'nastiness'" isn't one). What's not secure about SVM? What's not secure about VT-x? Why does Theo think that virtualizatio somehow has to imply legacy PC I/O emulation?
Ugh.
However, I see this more as if the virtualization layer actually sits under the OS layer, then the actual security for remote intrusion would be, first, Y/OS(X), THEN Y/V(X), where Y is the number of people with the knowledge to exploit each vulnerability. Thus, someone who wanted to exploit the system would both have to be capable of exploiting an OS vulnerability, and THEN also exploiting a virtualization vulnerability.
(And we're talking about remote usage, because we all know it's virtually impossible to protect a system from anyone who has direct access to the hardware.)
I understand that reality may not be quite as tidy, but it still seems like a virtualized system would be much more secure that a non-virtualized system, if only because the increased level of knowledge involved means a smaller number of hackers capable of exploiting both layers. What am I missing?
Lawrence Person (lawrencepersonh@gmailh.com (remove all "h"s to mail)
http://www.lawrenceperson.com/
You mean my strategy of running Windows inside of Mac Parallels inside of Pear inside a VMWare instance in a Wine bottle isn't the most secure, stable environment ever conceived? Sheeze. Maybe I should just get a Mac. :)
--
http://www.metagovernment.org/
GOVERNMENT BY *ALL* THE PEOPLE
And as there is no engineer that can develop hardware without security bugs, the only solution is to stay with insecurity!
Maybe Computers will never be as intelligent as Humans.
For sure they won't ever become so stupid. [VR-1988]
This sounds suspiciously close to his comments about journaling filesystems when asked why OpenBSD didn't support them (which boiled down to "journaling sucks, use softdeps instead"). OpenBSD has native support for exactly zero virtualisation schemes, whereas NetBSD has native Xen support (something Opensolaris is working on -if they don't have it already), FreeBSD and Linux both have support for kqemu and Linux and Windows both have support for VMWare, Virtualbox and kqemu.
For fuck's sake, OpenBSD can't even offer a modern version of WINE in their ports (the one they offer is from 1999, and is broken to boot).
So instead of fixing OpenBSD so that it has native support for running some sort of native virtualisation scheme, Theo does what he usually does -bitches, whines and blames the technology for the flaws in his OS.
"software which OpenBSD uses and redistributes must be free to all (be they people or companies), for any purpose they wish to use it, including modification, use, peeing on, or even integration into baby mulching machines or atomic bombs to be dropped on Australia" — Theo de Raadt
-- "At Microsoft, quality is job 1.1" -- PC Magazine, Nov. 1994
Theo's childish, condescending and pointless choice of language seems to undermine his credibility. Although he may be an authority on the subject, I think he owes it to himself - as well as the rest of the community he helped to create - to communicate in a more professional, civilized and respectful manner.
He's in the same bucket as Dvorak - who wants to listen to the little twerp?
Beware of the Leopard.
I do a lot of prototyping and testing out of scenarios with virtual machines. (40+ iterations for servers and client) Not all are complete builds as I do a lot of cloning. If you fire up a virtual machine that hasn't been in use for a while, you may need to spend time with security updates. Also if you didn't place or adequately configure virus protection and a firewall in an original clone you may end up with a number of machines with poor security. On the other hand cleaning up viruses is easy with my scenario, I just delete a current clone and go back to one not infected. (Assuming the virus is readily identifiable.
Use your head, can't you, use your head,
You're on earth, there's no cure for that - S. Beckett
I am not sure that the argument is right. Saying that virtualizion add possible security bugs is like saying that adding a personal firewall is adding fucntionaltity and thus possible exploits.
Virtualiztaion is more secure IMHO than current process isolation in most operating systems, but both can fail.
Theo's argument about security just proves the argument of linux about Security is "people wanking around with their opinions" is not unrealstic.
You have torealize that the alternative to virtualisation is getting an extra box with hardware or run 2 application on the same box. 2 machines is more secure, second solutions sounds LESS secure.
I have no idea with Theo's point here is. His statement is like saying, "Firewalls are programmed by the same people who write operating systems. If you think Firewalls have no security issues, then you're deluded, if not stupid." Therefore, Firewalls are useless and just increase complexity.
Virtualization, from a security standpoint, is just a firewalling method. It increases isolation between instances, and more isolation is ALWAYS good.
Sometimes it's best to just let stupid people be stupid.
It would be OK if you accessed the whole thing through an X-window running on a VirtualPC instance of Debian inside a Win 2000 shell.
I believe that he is working from the unstated assumption that the virtualization host and guest OSen all have approximately equivalent levels of security. If so, then virtualization does just increase the number of holes available for exploit. Rather like the way a RAID system increases the overall chance of a drive failing, because of using more drives. The difference of course, is that the RAID system is able to effectively isolate the failed drive, where a security exploit in one OS can potentially provide a path for breaching other OSen.
So his point makes sense given an assumption. But what if there were some OS "A" that was much more secure than some other OS "B". In that case, it is at least possible to increase the security of an installation of OS "B" by hosting it under "A", where "A" (and the virtualization software, which of course also needs to be much more secure), can protect it from a variety of attacks. So the question remains: what if? Hmm...
One unplugged from the network, with no applications.
If you want to do anything in this world, there is risk and generally, the greater the risk the greater any reward. So while he may well be correct, he's totally missed the point of well, everything. Leave him to stew in his paranoid fantasy world. I'm sure the NSA, CIA or whoever will be happy to use his skills.
Deleted
The ports tree is 3rd party stuff, not OpenBSD. Why don't YOU contribute instead of whining.
When a Port Is Lagging Behind the Mainstream Version
"The ports collection is a volunteer project. Sometimes the project simply doesn't have the developer resources to keep everything up-to-date. Developers pretty much pick up what they consider interesting and can test in their environment."
"If you really need a new version of a port, you should ask the MAINTAINER of the port to update the port....if you can send patches for this, all the better. To create proper patches, you should refer to the documentation on building ports."
-- "At Microsoft, quality is job 1.1" -- PC Magazine, Nov. 1994
Comment removed based on user account deletion
Theo tends to be cynical and pessimistic about just about anything that's got to do with security, and he's got good reasons to be... things that people push as security features turn out to be irrelevant or even actually dangerous a large proportion of the time. They're not batting 1.000, or even 0.500, by any means.
This doesn't mean that OpenBSD won't get some kind of virtualization support, it just means that he's being careful and conservative and letting other people be the pioneers. I think this is a good thing, on balance... you don't want to be pulling arrows out of your back because your secure OS decided to take you through unknown territory.
Yeh, he's got an emphatic way of putting things. You just gotta deal with it. Several years ago I asked him about stack protection and his response was eerily similar to this. A few years later OpenBSD enabled stack protection by default.
I think he's got a point, but he's comparing running separate computers to running separate OS instances on the same computer. If that's how you're using VMs, then yes, the resulting system is less secure overall... and for Windows that's often how VMs get used because Windows tends to make it unreasonably hard to run multiple instances of the same application on the same computer. If you're replacing less extreme isolation mechanisms on the same computer with VMs, though, then you're adding an extra layer of defence. Think of it as a hierarchy...
* Same application instance (eg, web server modules)
* Separate applications (running multiple instances of apache)
* User level separation (multiple accounts for the separate instances)
* File system separation (multiple chrooted instances of apache)
* OS-level separation (eg, FreeBSD jails and I think Solaris domains)
* Hardware-assisted software virtualization (VMware, Xen)
* Hardware virtualization (IBM VM "penguin farms")
* Separate physical computers
It might be argued that IBM's virtual machines should be lumped with virtualization, or that separate computers should be split from blades, and things like NAS and SAN complicate things, but you get the idea.
Theo's looking at the hierarchy starting at the bottom, and seeing a reduction in security. Other people are starting at the top, and seeing an increase in security. Both sides are correct, it depends on where you start.
As another person pointed out on the OpenBSD list, see http://taviso.decsystem.org/virtsec.pdf for Tavis Ormandy's analysis of various VM's -- attack methods were exploiting bugs in the x86 architecture as well as invalid I/O device communication.
Now, if I want to save money, I could combine both onto one box without virtualization. This is the least secure way to do it, as if someone compromized the mail server, they would only need to overcome the user-level isolation to then gain access to the mailserver.
If I want a setup that is not quite as secure as the first option, but much better than the second, I could create two virtual servers, one for the webserver and one for the mailserver; this way, if someone compromised the mailserver, they would need to overcome both the os-level protections and then find a hole in the virtualization isolation (which, from what I understand, hasn't yet happened with paravirtualized xen- HVM xen is much less secure.)
when you are running a paravirtualized xen setup, the big thing to be concerned about is the Dom0; you should never have an external IP on the Dom0, as if the dom0 is compromised, it is all over.
This started when a guy on his message list suggested that OpenBSD get a Xen port and believed "Virtualization seems to have a lot of security benefits."
Theo responds by insulting him, then downplaying virtualization as if it were some kind of toy. You've seen something on the shelf, and it has all sorts of pretty
colours, and you've bought it.
That's all x86 virtualization is. Further down the postings, you get a better idea of what Theo's opinion of VMs. If people were saying:
"Yes, it increased hardware utilization, and the nasty
security impact might be low"
it would be fine.
But instead we have many uneducated people saying:
"Yes, it increased hardware utilization, and it improved security too".
And that's complete and utter bullshit. All that can be taken from this silly "news" here on Slashdot is that NOBODY had better use security as an excuse to weasel something past Theo de Raadt. He apparetnly has a very good nose for both weasels and security.
After scanning that thread, I was quite amused. This is a classic example of everybody talking about something, believing they are talking about the same thing, but. . . not really. Nobody notices it because nobody states an essential premise.
We have one poster, "Lee", who states that he feels that Virtualization improves security. . . but he fails to define his point of comparison - more secure than *what*? We have other posters, Theo among them, who state that they feel Virtualization decreses security. . but they also fail to define their point of comparison - less secure than *what*?
After reading through the posts, I think what this argument comes down to is that Lee feels Virtualization improves security compared to a bunch of users sharing resources of a single computer (e.g. a shared web host, where there is a single instance of Apache on a single instance of whatever OS, and they each have their own virtual websites in seperate directories [as one example]). Particularly where you want to give those users a relative amount of freedom. From his basis of comparison, he is possibly correct (though I suppose the argument could be made that the VM is still not really any safer than a server that is properly setup to be shared by multiple users).
I think that, the argument that Theo, et. al., are trying to make though is that multiple physical servers can provide guaranteed isolation that a single server running multiple VMs cannot do. So, I think, the basis of comparison for them is that one server is not more secure than multiple physical servers, and that if you are looking at software-based security on a single machine, a software hypervisor is probably not any more secure (and, at least theoretically could be less secure) than the Operating System is. I dunno if that is true or not, as I do not have any expertise in these matters. But, in any case, I found the thread to be a fascinating example of what, appears to me, to be miscommunication, because people feel it's *obvious* what they mean, when they appear to have omitted a necessary piece of information, to me.
the cause of the Famine needs to be shared by the idiotic British
government at the time, and the greedy bastard landowners, as
well as the fungus. If the Irish had been free to BUY FOOD FROM
ABROAD not that many people would have starved.
Reduce, reuse, cycle
Theo is like the fox in the Aesop fable.
."
That fox saw some tasty grapes hanging just out of reach on an arbor. Try as he might, jumping and hopping as high as he could, that fox just could not reach those grapes. So the fox trotted away muttering to himself, "Those grapes are probably sour anyway . .
When Theo or his pet project has technical problems with some concept or hardware situation, instead of solving the problem with his OS, he squawks and rationalizes that the hardware/software/whatever is "stupid" and "doesn't work right". Theo is the living example of what Aesop illustrated so many years ago in the fable "The Fox and the Grapes".
Theo de Raadt argues that it's more secure to put applications on separate machines than to consolidate them into a single machine.
L. V. Lammert very inarticulately argues that having a VM provides more security, because otherwise, you're not going to put applications on separate machines, because it's too expensive.
http://outcampaign.org/
Here's the first truth of security: your ability to secure a system is INVERSELY PROPORTIONAL to the size of the interface to that system. Every interface point is a potential attack vector, whether direct (an attacker can exploit the interface) or indirect (something outside your control is loaded at interface A, then an attacker at interface B causes A to exploit something). Most security products try to reduce the size of interfaces (e.g. a firewall limits the number of open ports, then further excludes some types of traffic from those ports).
Look at a general purpose operating system kernel. There are hundreds of system calls (direct attack vectors), hundreds more driver interfaces (indirect attack vectors - driver interfaces are privileged and thus drivers must be bug-free), a few thousand more configuration points (Windows Registery, Linux /sys and /proc trees). Add the libraries that make up the rest of the operating system, and the number of APIs has exploded to thousands, if not tens of thousands.
Now look at a hypervisor stack. The hypervisor::guest interface is the CPU instruction set (extremely well documented and easy to programatically verify, especially when 99% of instructions can be verified to have no side effects!). Much narrower interface than a general-purpose API. The driver::hypervisor interface is narrower too, since the hypervisor only uses a lower-level interface (e.g. Xen's block device interface, VMware's SCSI interface) that happens to be simpler and better documented. Configuration API is smaller, since it only needs to manage virtual machines, not every possible combination of user-level program and device.
It's the old microkernel / monolithic kernel debate all over again, where a hypervisor is a microkernel and a general-purpose OS is a monolithic kernel, and the performance loss is small enough that companies are using it in production today. Microkernel have advantages in being easier to secure, more robust in the face of bugs ... monolithic kernels are faster. Is the smaller API (and increased security) worth the loss in performance?
Here's some security thoughts, based on actual experience with virtualization bugs.
A witty [sig] proves nothing. --Voltaire
When did segment registers and page tables quit working? I guess I must have missed that errata!
This is my sig.
We are talking custom in-house apps, not running DNS and a web server on the same box. I am and always have been a UNIX admin, I was taught this method, and I've seen it done in every place I've worked.
Dumb AC fanboi. Pick a better reason to dis Windows.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
Virtualization increases security against nastyware that isn't aware it's running in a virtual layer. There's nothing to stop nastyware that's written to penetrate (or even exploit) virtual layers. So, yes, virtualization increases security. But not by much, and not for long.
...the main reason I want to run a VM is to use existing Operating Systems that are written for x86 hardware, and I'm willing to bet that's also the main reason others use VMs, too. If you change the architecture of the HyperVisor, now you have to write OSs for that, and say hello to another seven versions of Windows plus thousands of additional *nix and *bsd distros.
For application testing and tinkering with various operating systems VMWare and the like are a godsend.
For hosting providers given the current state of things I can see the advantages of advertising 'root' access to a virtual machine, this is useful but not ideal.
For certain high reliability tasks the ability to pause and migrate VMs between systems is compelling.
From my POV the 'buz' around virtualization does not justify what the market demand should be.
Virtualization is great for CPU and Memory manufacturers but for consumers the better solution is ususally improved security access controls/jails/etc that allow isolated execution of untrusted applications. There is no reason this should require execution in a virtual machine.
Virtualization is basically a gross simple hack rather than solving a problem at the OS and application levels where they belong.
VMs do add some interesting security twists...especially giving people root access they can have lots of fun with network access (ping -f works:) (L2/L3) and mess with the operation of other virtual machines by configuring bridges and stealing data transiting the network - Requirement for Security for access to external resources is one reason I believe the VM approach is fundementally incorrect from a security POV.
However I disagree that its harder to write a secure VM environment than a secure OS. Popular VMs benefit too much from economies of scale and overall less complexity when compared with the security models and escalation opportunities of modern operating systems.
I think Theo is making some unstated assumptions. If you assume that in the absence of virtualization would result in the different virtual machines each being their own server, then if root level access on one VM would allow you to take over the host OS, then the security of each machine on the server has been reduced to the security of the weakest machine on the server. And since it is his assertion that virtualization would increase utilization, it is reasonable to conclude that there is a good chance that that is the assumption he is making.
The counter assumption is that in the absence of virtualization all the services that would have run on the VMs would just all run on the same server, then virtualization can increase security because no one service may constitute a top to bottom root exploit but some combination of them may. Of course adding the extra overhead of the VMs potentially decreases the utilization your services get from the machine. However most of the time when security is discussed it is this scenario they are talking about.
And of course the reality of what happens in the field is that there is a little of both going on. You have both server that did too many things being broken up into VMs, and clumps of smaller servers being replaced with a single larger capacity VM server.
[disclosure text="I work for a company that sells virtualization."]
Theo's expertise, and indeed that of the entire OpenBSD project, is in the realm of provably correct security. Virtualization adds yet another layer where something can go wrong. Sure there are and will be bugs. We're finding them and fixing them, just as we've always done. From an absolute security standpoint, Theo's right.
Of course, most businesses couldn't care less. Businesses don't view security as an absolute thing, because human factors make it generally impossible. Businesses view security as a risk, with associated probabilities and costs, worst-case scenarios, likely scenarios, mitigation strategies, and ultimately, diminishing marginal returns. For businesses using virtualization to consolidate systems, it generally reduces risk because it makes it easier to implement policies that mitigate human factors.
To be precise, virtualization *technology* decreases security, but virtualization *solutions* increase security, at least when done well, which is much more practical than the technical absolute of "done right".
[/disclosure]
There's no failure quite as dissatisfying as a complete and total solution to the wrong problem.
I'm always amazed that virtualization on x86 works at all. The architecture didn't support it, and VMware dealt with that by dynamic code patching. Then Intel and AMD both added some hardware support, but 1) incompatible between Intel and AMD, and 2) single-layer (you can't run a VM on a VM on a VM, unlike IBM mainframes, where that works quite nicely. And then there's the issue that you now have two levels of scheduler, which may or may not play well together.
But those aren't the real problems. From a security standpoint, a VM running a whole operating system is an overly large security domain. It doesn't even contain, say, exploitable PHP scripts on a server, let alone a rootkit. In fact, almost any exploit that will work on a standalone server will work on the same server inside a VM, and can cause just as much trouble on the net. Now you can have multiple corrupted VM partitions!
What we're really seeing is that server virtualization is a way to unload security worries from the server farm operator (be it a hosting company or an in-house operation) onto the user. If the server operator just gives the user an empty partition and takes no responsibility for maintaining it, it's cheaper to be a server operator. Server management is easier. But it's no more secure from the standpoint of network, user, or data protection. Too much code in one security box.
Ugh, I hate it when I have to agree with Theo, but in this case, I think he's got a point. Hypervisors will have bugs, and some small portion of those will lead to security holes. I don't care how carefully the code is audited, when you're dealing with handling an architecture this nasty, there will be bugs. Because of the smaller amount of code and services that a hypervisor provides, it should be easier to get all the big bugs out, but I'd be willing to bet that some small ones will hang on, undiscovered, for some time.
As far as VMs and security, there are two types of security - defense against the malicious, and defense against the retarded. While VMs may not add much in the long term against the malicious (and may even expose more risk), I'd argue right now they're reasonable tools of isolation, until virtualization becomes mainstream and crackers get wise to exploiting the host machine.
They are effective, from a defense against the malicious standpoint, for isolating old platforms that you continue to need, but can't be exposed to the world. We have a proprietary tool that only runs on NT 4 (we've tried 2k, XP, etc. and no dice) that we absolutely must have at work. NT4 is no longer patched when security issues are found, and there are no longer drivers for the hardware we've had to move it to. So we run it in a VMWare VM that has no network access. Problem solved.
I'd argue that VMs are very effective on the "defense against the retarded" side. We have a shared departmental webserver here my job. I'm the main admin in my spare time. However, when they merged a bunch of groups, they made us share the box with them. Thus, management mandated they be allowed to change things as root to fit their needs - adding users, resizing LVs, fiddling with apache, etc. They kept fucking up the box and breaking our stuff. So eventually I loaded up Xen and gave them their own VM that they could break in any way they wanted. Tada - their virtual box is always screwed up, mine stays up all the time, and management is happy because we didn't need more hardware.
Arguably, the above also fits within the greater utilization purpose that I see being the big driver for virtualization. Realistically, most production boxes are far oversized for what they do. If you could stack virtual boxes on real iron to get better asset utilization, that's going to be the driving force for business, as it's a simple, quick way to reduce costs.
You're Broadcasting an IP Address!!!!
Seriously. It goes a bit too far to suggest that virtualization decreases security simply because of the added accessibility or mode of operation. But it's not "untrue" either. But to ignore the benefits of virtualization because of unknown but presumed added risk would be like securing your server by removing the networking from it. Yes, there are risks, but they must be balanced against the benefits.
And as for attack angles against a virtualized server? Well, you'd just about have to compromise the host first anyway and by that time, the game is already lost. (Now if someone were stupid enough to make the virtual machine's management interfaces available to the internet, that's ANOTHER problem) But otherwise, all the same steps needed to compromise a virtual server are pretty much the same as compromising a server running directly on the hardware.
I know that this will be probably marked as flamebait, but I have to at least partially agree with my fellow Canadian (Theo). To me, all forms of virtualization are never going to provide 100% security. To me, virtualization is a lot like using a NAT in a network: For a person smart enough, the 'security' in a NAT is easy to get around (check out http://www.google.com/search?q=+site:www.merit.edu+nanog+nat+security for evidence).
However, what Theo skips over is that both NAT and virtualization are great for keeping out the riff-raff. At least on my home connection, the network traffic outside of the NAT is at least 2 packets/second of bad stuff (measured via FreeBSD 7.0's security log/emails), and inside the NAT the traffic is less than a packet/day of bad stuff (when all Winblows machines are off). I believe virtualization can do the same and keep virii/spyware/bugs/etc. contained and better monitored.
So in conclusion:
In theory, I believe Theo is right.
In practice, virtualization is probably a good thing for enterprises, just as NAT is a good thing for home users.
I am hoping that people will stop yelling at each other and realize that Theo likes everything to be practically and theoretically secure, and most of us just care that our machines are "practically secure enough." However, as botnets perhaps become more profitable, it seems that they are getting smarter, and have been using multiple exploits for quite some time, so Theo might have a point.
The technology itself does not add or subtract security. It all comes down to how you use the technology.
A lot of people use virtualization to combine physical machines into one. They do this for utilization, and not for security. I can see how this might make the machines less secure, since if you hack one virtual machine, and break out of the hypervisor, you now have 20 (virtual) machines hacked.
However, there is another common way to use virtualization. A desktop user might surf the web in a virtual machine. We have been told that this enhances security. It does, using the classic "layers" solution. Not only does a hacker have to exploit the user's browser, but he also has to break out of the hypervisor. The hacker doesn't know what hypervisor he needs to break into beforehand, and his time is limited to the amount of time the user spends surfing the web. Most likely, the exploit was automated, will not break out of the hypervisor, but instead die when the hypervisor is shut down, and the hacker will never even notice. Compare that to having your machine compromised as soon as a hacker exploits one of the flaws in firefox.
Write your own Choose Your Own Adventure. http://www.freegameengines.org/gamebook-engine/
And it's one problem you can avoid by not using virtualization.
Theo's point is a very simple, obvious fact. Why is this necessary to even argue about this ? He is not saying virtualization is useless. He is not saying it has no practical uses in the real world. He is just saying it reduces your overall security by introducing new potential attack vectors.
One concrete data point: a VM running under QEMU 0.9.0 can escape the virtualized environment by exploiting buffer overflows in the code emulating the virtual NIC. If you run under a VM you are vulnerable, else you are not. Remind me why you are arguing about this again ? See http://taviso.decsystem.org/virtsec.pdf for a bunch of other vulnerabilities recently discovered in VMware, Xen, QEMU, etc.
It takes more calories to produce a child then you get from eating them.
Forget producing them. Have you ever tried keeping up with a kid who is trying to run away from you? You'll burn more calories trying to catch them than you'll get from eating them.
When our name is on the back of your car, we're behind you all the way!
Check out Parallels .mem memory dump files written out on suspend.
Read the whole of the memory of the emulated machine. Whatever it contains.
I have to pipe in here about the fallacy of the irish potato "famine" and the rewriting of history by the English - even if it is off topic a bit.
There was no "famine" in the true sense in Ireland. Read the REAL history.
This is what happened -
At that time in Ireland there were actually many many different crops still being grown, sheep and cattle were being raised. There was lots of food to go around - even plenty of it. What was going on was that the English - being the great oppressers that they were (and still are) at the time were harvesting all this food and shipping it via protected and secure means to their own tables. The only food they did not cart off for themselves was the potatoes. They basically stole all the other food that was being produced at the time for themselves - leaving the irish with nothing to eat but potatoes.
So yes a blight did hit the potatoe and many people starved.. but it wasn't for lack of enough food being produced.
--
You can ping 1208941928
> CAUTION: flame war ahead.
...)
There doesn't need to be a flame war, because in this particular instance Theo's argument has a gaping hole in it. Consider the following two system architectures:
1) An ordinary multi-function Unix-type system which also runs a non-trivial component that is exposed to the world (all non-trivial components have bugs, as Theo is right to point out, and hence are attack vectors).
2) A machine running 2-guest virtualization, in which the non-trivial component runs in one guest, and the rest of the functions run in another.
Now consider what happens when the world-facing component gets compromised, and by one of many methods (because sysadmins are fallible) the attack gets promoted to root privilege. Security has failed in one guest, but has it failed in the other? Not necessarily, depending on whether the sysadmin has made repeated blunders and not just one. (Eg. a fool might be keeping ssh keys on the public-facing guest
In this scenario, the isolation created by virtualization has given the syadmin an additional bulkhead against his own fallibility, and that is worthwhile for security, not only for better hardware utilization. The partitioning of the application and O/S space has reduced the cross-section of software open to attack.
Theo's argument also doesn't bear scrutiny at the hypervisor level, because while an O/S in dom0 is just as fragile as the one in domU that runs an exposed application, the instance in the hypervisor isn't exposed to attack. Theo seems to miss the distinction between endpoint fallibility and fallibility in the conveyance and resourcing that is done by hypervisors. They're different.
I like Theo's hard stance on security, but on this issue he's handwaving.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
Tits... sorry, exploit examples, or GTFO.
In other news, Virtualization reduces DRM from "pipe dream" to "joke". Virtualization is the ultimate man in the middle. It sees everything and controls you're ability see to anything. A hardware virtualized host is living in "The Matrix". RDTSC -- virtualized, CPUID -- virtualized, RTC -- virtualized, page tables -- virtualized, memory access -- virtualized at will, LAPIC, IOAPIC, PIC, PIT -- virtualized, interrupts, breakpoints, and faults -- virtualized, motherboard -- virtualized, PCI/PCIe chipset virtualized, BIOS -- you guessed it...
Essentially what used to take a $1M in logic analysis equipment is all there for a free download of Xen.
Unless a DRM requires access to a secure off-host authN/authZ system it is naked for all the world to see.
Actually I think he might argue that chroot'ing has been audited a lot more than any hypervisor and might be the more secure approach in many/most instances if it's set up correctly.
Some very well informed people might reply that "chroot is not and never has been a security tool."
While I love Theo when he's being outspoken (here he's clearly right, on principle -- humans write buggy software, and adding more code in this fashion makes it worse because there are more, orthogonal vectors of attack...) it is still good to back things up with empirical data, as the parent post does.
- "History shows again and again how nature points out the folly of men" -- Blue Oyster Cult, 'Godzilla'
Finally, I see someone gets it! The issue, here, isn't whether a vulnerability in software on one VM could lead to a compromise of another. The issue is that the VM *itself* maybe insecure. And an exploit in the VM can lead to the entire machine, and all VMs on it, being compromised. *That* is the additional layer of software Theo is referring to, software which may be buggy, and is most certainly subject to attack.
What Theo is really saying is that virtualization is insecure until OpenBSD has a virtualization system integrated into the kernel in 2020.
Theo criticizes the security of every new technology until it's implemented into OpenBSD (I bet a quick search would show 100's of examples of this, in fact I think he said something similar about SMP when Linux first implemented it), it's part of his jealousy of the gnu/Linux system having more developers and resources than OpenBSD and his belief that only OpenBSD is secure.
He's such a bright guy, why does he continually have to act like a spoiled child?
For client machines, using a VM sandbox to run your web browser could keep the host from being compromised by malicious code execution. I can't imagine how that could make browsing less secure.
For servers, what I'm really looking forward to is a hypervisor that trips on security events such an attempt to remount r/w a filesystem that's been locked down. It would respond to such events by forking off the VM into an ad-hoc honeypot/tarpit.
[100% ISO 646 Compliant]
SVM, ERGO MONSTRO.
In other news, researchers have recently concluded that levels of personal privacy increase inversely to the number of people you live with!
In other words, THANK YOU CAPTAIN OBVIOUS!!! WE WOULD HAVE NEVER KNOWN!
Consider yourself spoken to.
But that's just security through obscurity. You're kind of assuming a probability-based model, where the goal is to lower that X percent chance that some random hacker might wander in and decide to go at your systems. What if you know for a fact that somebody is already out to get you? What if you're in a highly competitive industry that's prone to industrial espionage? What if you're the United States government? In such cases, the fact that exploiting a vulnerability is difficult to do, and therefore not many people have the required level of skill, is not relevant. You can assume that your enemies will find those people, and then all that "much more security" instantly vanishes.
Breakfast served all day!
The issue is that the VM *itself* maybe insecure.
... you can't target them.
You're just not getting the problem with Theo's position at all. Of course the VM is insecure, because all complex systems have faults. That is not under dispute.
But those faults (WHICH MUST EXIST in the VM and hypervisor) are not exposed to attack in the same way that a guest operating system is. The hypervisor O/S is almost entirely transparent to the attack vectors, and that's the issue that Theo conveniently ignores to give himself a platform.
Theo's premise would be valid only if all VMs including the one at hypervisor level were endpoints so that they would all be equally at the mercy of attacks. But they're not endpoints. The hypervisor is just a conveyor and a resource manager and simply can't be the target of attacks against an endpoint guest, and traffic destined for endpoint guests doesn't go through or into or anywhere near other VMs that run systems and apps that are not endpoints.
When you shoot at a window, the window won't break if it has no glass to intercept your shot. That's what hypervisor VMs are like
The one place where Theo's rant makes a valid point is when he says that new code has more bugs than well tested code --- yes, that is undoubtedly true, and "new code" definitely includes virtualization code. But it's not specifically a point against virtualization but against all change, and in favor of stability and the slowing down of development. If that had been his point then fine, but when he singles out virtualization then he's just off on another "Theo rant".
It's a pity that he goes off on these rabbit trails, because he's good at what he does and shouldn't need to. But boy does he love a platform on which to rant, even when his point is pretty thin.
As an ex-developer for VMware ESX Server I can tell you that the system is extremely complex. There is a complex kernel for running virtual machines, and a whole lot of fancy difficult to grasp stuff going on in the Monitor. It is apparent that assuming that a very large and complex piece of relatively new software is bug-free and secure is a dishonest sale's pitch. Especially when you compare it to the simplicity of using separate under-utilized machines or using simpler, more mature and much better understood chroot jail environments on BSD.
Let's take my situation. I boot Linux, fire up a virtual machine manager (VirtualBox), start WinXP to run just those Win apps I have to have - which don't need Internet access, so I disable that most of the time in XP. My internet stuff is all done in Linux.
Question: am I now running XP in a more secure fashion versus using it for everything? HELL YES. And I would argue this is more secure for Linux itself, because I don't need to run Wine with it's own set of security issues.
Jim
Theo's OS doesn't have Xen so therefore Xen is worthless crap.
Bluepill and Vitriol are irrelevant when you already run an hypervisor on your system.
;)
This is something people have a lot of problem grasping, but you are NOT installing a rootkit hypervisor on a machine that already is running an hypervisor. Sure, there have been some security issues (Red Hat fuxx0red pretty bad with Xen, allowing a file in a guest to r00t the hypervisor, but that was a Red Hat problem). Hypervisor are *small* compared to full OSes. They are *way* easier to secure than a whole OS, even if this OS happens to be OpenBSD.
People crying "but Bluepill or Vitriol can r00t your box" conveniently ignore the fact that it's not working when your machine is already running an hypervisor...
And they'll never be able to prevent being detected by a timing attack (that's not an opinion, it's a fact).
So please please... Enough paranoia. Go install Xen and see how complicated it is to run Windows at reasonable speed + all the hardware working fine and now you come back and you tell how Bluepill or Vitriol will be able to stealthy r00t a box. Note that I'm not saying it's impossible to r00t an hypervisor but that's it's impossible to install a stealth hypervisor that can then run another hypervisor that can run guests inside VMs (on the current generation of CPUs it's simply impossible: and good luck emulating VMs instead of virtualizing them and pretending it's still a 'stealth' technology... I can't help but laugh about that one
In addition to that from a security point a view it's not just because there's an hypervisor that the guest OSes are more secure: it's also because it's way easier to re-image them, to transparently move them from one machine to another, to scan them from the host, etc. that they are more secure. There are a lot of reasons that makes a virtualized OS being a smarter choice from a security point of view.
Sure, there can be hypervisor-exploits. But will they stand the Tripwire integrity check you'll run on your hypervisor + VMs that you just took offline and are running by mounting the drives read-only from another machine? Remember that it's *trivial* to do because you just migrated your *running VMs* to another hypervisor.
I'm a big fan of Theo, but sometimes he doesn't tell the whole story.
Most people commenting in this thread have no experience running an hypervisor, live migrating VMs and running integrity checks on read-only hard drives, so take their words with a grain of salt...
As a final note, wheter you like it or not virtualization as so many advantages (security, when done corretly, being one of them) that it is here to stay.
But they're rare, and the current raft of hypervisors don't rate as being possibly secure.
One area in which they fail to pass muster is in regard to their layering. From the DEC paper:
One noted security practioner (not me) has opined in private correspondence that, while "[c]learly much more than this is required (as reflected in the rest of the DEC paper), but without such a strict layering a VMM design cannot be considered a serious candidate, and is probably not worth spending time and energy on. The precise number of layers will depend on the particular design, but consistent experience over several decades indicates that for the kind of functionality in a VMM something on the order of the 16 layers in the DEC design are essential to "minimizing the complexity". A small integer number of layers is an immediate tip-off that something "smells rotten" in the design."
Such minimization of complexity is necessary to support the formal modeling and analysis that would allow you to verify that, for instance, there is no unnecessary code in the hypervisor that might be exploited by an attacker (whether deliberately inserted or not).
But, in today's world, who cares? These lessons were learned decades ago - they couldn't be relevant today...
...sarcasm off
Not pretending to read Theo's mind, but my own investigations into Xen et. al. over the past few years for work simply pointed out that the holistic security story includes not just the OS, CPU, and MMU but all the I/O devices (particularly every bus-mastering capable component, but others too). Worse yet, the pressure for better performance is leading to many compromises such as multiple "driver domains" in Xen, and other paravirtualized drivers that give more I/O hardware control to guests. The problem is that the PCI bus is not really a safely virtualized communication environment, so giving control of DMA to a guest often means in practice that they can corrupt other guests and/or the hypervisor at will. It is cooperative safety, not mandatory protections. They do not really have a safely isolated virtual PCI, but all play in the same hardware security domain, which is furthermore as secure as the most poorly behaving attached hardware.
A similar problem exists for other things abused as security devices, such as VLAN. It is telling that published documents from network equipment vendors warn against using VLANs in an attempt to isolate distrustful hosts, yet people blithely go ahead doing this anyway. There are too many potential holes in the entire distributed system to believe it is secure.
The risk of virtualization hype, as pointed out already in this discussion, is in naively thinking that it is safe to consolidate guests with very different trust profiles onto the same host. It is certainly safer to keep them isolated on separate hosts. Now, the question of consolidating guests in an equivalence class of trust is different. The only real observation here is that virtualization makes a larger system with more components and more complexity that could go wrong, from a statistical perspective.
...virtually impossible.
Theo does has a point. I've personally caused a VMware host to crash from a vmware guest by changing the date on the guest O/S to something invalid. I doubt that's exploitable other than as a DoS attack but given the weird sort of stuff that's needed to enable copy and paste and other "magic", I won't be surprised if there are more exploitable bugs.
;).
:). It's not going to be so easy to break out of an Apple II emulator and get the Windows machine it's running on 0w3ned.
But, VM technology is old stuff. It's just the x86 people are reinventing it and sometimes poorly. Heard something like this from one of the "new VM" people, "If we have VM problems we just go ask the IBM guys what they used to do in the old days to deal with that"
Sure there's probably exploitable security bugs in many VM implementations. But that does not mean there will be exploitable security bugs in all VM implementations.
You could run a pure x86 PC emulator and I think it'll be quite safe, though rather slow
Can one really claim that it is impossible to get a virtualized machine as secure as an emulated machine (in practice anyway)?
Ideally we should run apps on dedicated boxes. Virtualisation is good for saving money and for some "virtual" security against script kiddies, newbie crackers, or your own users (or yourself). VMs can't defend you against real crackers, and in fact they will make you less secure. Do it for the money or for protecting against script-kiddies and users, but not for real security. Always use multiple boxes for various apps if you can do so.
whether to use my last mod point this round to mod your post overrated or to respond directly.
Well, yeah, Theo apparently mentioned the penchant for programming holes.
But, no, that's not his argument.
Partitioning isn't perfect in hardware designed for it. The PC is not designed for it. Do you think the software can take up the slack?
Think about iNTEL's first attempts to make CPUs that handle threads better. With all the fufarah about hyperthreading, I don't remember what iNTEL called the stuff, but the only fix was to disable it, which was a no-brainer because it really didn't speed things up worth noticing. PCs have lots and lots of such good-ideas-that-wasn't.
So, you have several such bugs on your CPU. One of your hosted OSses is set up to accept the security risk for some (possibly valid on separate hardware in a separate segment of the LAN) reason. Something core dumps in that hosted OS and some of the garbage in the unallocated areas of the dump contains (!) private keys from some other hosted OS that the admin _thought_ was properly patched.
Lots of complaints about hand-waving, but do we have to burlesque?
is not much of a security argument.
is what management is thinking.
do you think you are emotionally mature?
Now I wish I hadn't posted to this thread.
(Maybe I should do like so many others and get another account just for accumulating and using mod points.)
VMs should be restricted to the soft inner arena?
(Many others have suggested this already, I know.)
But, yeah, the systems where they run those proofs are most likely to be either
(1) enclosed within a sandbox so they can qualify their results (formal proofs can't deal with unscheduled input like a real attack.), or
(2) running as honeypots in the real world.
Neither of the above is production networking equipment.
Elsewhere, lists of vulnerabilities to do exactly what you describe have been mentioned.
And I suspect, as the first comment to your post points out, that you misunderstand the level of virtualization being obtained in software VMs.
I'll call you on my cellphone when I'm ready for you to open it for me.
;->
Kind of you to volunteer.
Hey?
you're being ignorant, and we don't particularly feel like spoon feeding you the information you're ignoring.
That's the way it is over there. The people who contribute to openbsd don't have to be told where to look for things. If the person to whom Theo was responding is really interested in contributing to that project, he's going to have to learn to do his own research.
VMs can be useful for many kinds of end-user applications. Careful use in end-user applications may actually lead to improved security in some cases.
They are also useful for _research_ _into_ security (testing hypotheses about provable systems, implementing honeypots, etc.) People who can do this build their own tools anyway.
You don't want to put your firewalls, DNSses, watchdogs, and routers in VMs.
Yeah, that means that I think that when we finally get to the point where it's recognized that each house needs its own full-blown LAN, we are going to have to put at least three processors in the "broadband modems" that now may or may not come with firewall:
One processor for firewall.
One processor for DNS.
One processor for watchdog.
Each of those could probably be implemented in a low-power, single-chip semi-custom, complete with on-chip flash for logging and setup. Perhaps all three could fit in a single package, as long as the CPU and flash are kept separate.
Some semi-custom processors, I believe, have sufficient cpu support for virtualization that one might be tempted to implement better memory management and combine them. But now we are talking about a completely different world from VMware, XEN, QEMU, etc. And I still wouldn't do it, because, even though none of those need to be fast, they all need to be responsive. And I would prefer to use separate RAM and separable seek and read/write circuits for the flash, rather than add the complexity to the virtualization monitor required to properly enforce separation on the non-volatile store. These kinds of CPUs are not that expensive.
And, of course, if one is not careful when designing such a device, it is really easy to introduce hardware vulnerabilities, (I've mentioned sharing RAM and flash as one temptation.)
Google funded research proved that. So it only then goes to say, that if virtualization software adds vulnerabilities to allow jumping from guest to guest or guest to host, then your overall security across all guest VM's and the host, is only as strong as the weakest service provided by any one of the VM guests.
But hey, this issue must not be real, since Theo brought it up. Right?