HVM is not is not faster than PVM. You might be confusing PVM for the software-driven virtualization that VMWare/QEMU/Virtualbox/etc. used prior to the introduction of VT-x.
From my understanding, paravirtualization works by running a modified kernel in the guest so that knows it's really in userspace of the host OS (Ring 3) instead of where it would normally live (Ring 0). VT-x basically adds additional namespaces that allow guest operating systems to run in Ring 0, but this process adds some overhead. And for some reason I've noticed this leads to some massive slowdowns in some applications, especially anything that does a lot of FLOPS.
I'm not an expert in these matters, but everything I've seen says (and my own experience verifies) that paravirtualization is generally faster than HVM. I'm running Qubes right now and all of my paravirtualized applications respond normally. Guest OSes in VirtualBox with VT-x enabled (on this same exact hardware) could get quite sluggish.
I've never used Docker but it is (so I gather) geared largely towards LXC virtualization. This is radically different from either PVM or HVM as it is basically creating multiple namespaces within a single kernel. It should be faster than either HVM or PVM, though still not as fast as a chroot. Depending on your goals, I don't necessarily agree that it is "best suited for application-oriented virtualization." It's fast and it removes the problem of RAM management, but from a security standpoint the compartmentalization isn't nearly as airtight and you lose the flexibility of being able to utilize different kernels. The overhead of the alternatives can be non-trivial, but silicon is cheap. Like I said, it depends on your goals.
All of our commonly used, general purpose, end user operating systems suck. Separation, compartmentalization simply isn't there. Everyone knows the advantages of proper compartmentalization: easy portability, unparalleled security, a mighty robustness against system-crashing errors and reboots, configuration flexibility and avoidance of dependency conflicts, the ability to easily prioritize system resource usage, the ability to roll back snapshots without rebooting the entire system, sharing or cloning base images, etc. But our OSes can't do any of this out of the box.
So people slowly added it back in, first with file permissions and then mandatory access control (Apparmor, SELinux), chroots, FreeBSD chroots, traditional virtualization (at first entirely software-based, then hardware-assisted), PVMs, LXC, Unikernel, and probably a dozen more approaches I'm unaware of. All of these things have advantages and disadvantages, but the important thing to keep in mind is this: they're all trying to fix something that is fundamentally broken. And ultimately all of the best solutions involve figuring out different ways of emulating multiple OSes because the applications we want to run only run on "modern" OSes... none of which are built around a concept of strict compartmentalization.
Enter the Genode aficionados and the hordes of older microkernel partisans who've been screaming about this stuff for decades. They will scoff and say a bunch of undeniably true things... but of course, last time I checked everyone who actually used Genode for everyday tasks was using it to boot VirtualBox or something similar. It's inescapable: we have a bunch existing applications we want to run and we simply can't run it without (in some fashion) running the crappy OS it was designed to run on.
And so we have a dozen different varieties of duct tape to choose from, each with their own advantages and disadvantages. It's interesting. It's fascinating. Currently, I'm leaning towards paravirtalization as the sweet spot of performance and flexibility and security (Qubes OS was the decisive influence on me here), but LXC is faster and simpler and Unikernel, while not simple, sounds like it could be very fast indeed. And I'm sure the dirt-simple appeal of vanilla HVMs will endure regardless of the advances Docker makes.
It's an interesting debate with no single right answer because of the long list of different pros and cons. But it's also interesting to consider how and when all of this is likely to be rendered moot. At some point, a new OS (probably microkernel based) will finally gain enough traction and get enough native end user applications to make it usable without in any fashion having to emulate another OS on top of it. People will start porting other applications to run on it, and eventually the Windows and Debian and Red Hat and OS X and Android and iOS of today will finally be abandoned as the insecure, unstable, kludge-ridden clusterf___s they are.
The question is... will this revolution be driven by an OSS project? Will the microkernel world finally get their Linus? Or will this be the ultimate proprietary walled garden, a massive investment by Apple or Microsoft or Google, some heart-stoppingly expensive super secret project to finally offer their customers a UI that never gets laggy or unresponsive, a core system that is virtually immune to malware, app developer APIs that rarely need changing, real time performance when necessary, and fine-grained app permissions that actually work properly?
Unfortunately, I think my money has to be on the walled garden, and even more unfortunately I'd have to say that Apple stands the best chance of pulling it off given their strict control over hardware and their titanic cashflows. Maybe Google will get there first and maybe, just maybe someone will be able to convince them to open source it.
It's fascinating to watch all of these brilliant people come up with these interesting and inventive kludges. And useful, undeniably useful in the here and now. But at the end of the day, I can't help but think... dust in the wind.
The layers add abstraction, compartmentalization, portability and the ability to roll back snapshots without rebooting. If you understand virtualization then I'm sure you can understand how this can greatly increase security and robustness in general, at least in principle.
Now, if you don't understand why someone would use an LXC-based solution like Docker instead of a fully virtualized HVM machine then you probably haven't seen them in practice. They are native speed, and you don't have to mess around with inefficient memory allocation. The performance difference is critical in many cases and even when it isn't, it's still damn nice to have everything being as smooth as normal. Unikernel is a different approach, but offers similar performance advantages over traditional HVMs.
As I mention below, paravirtualization is also pretty damn fast and is more secure and flexible compared to LXC solutions since the kernel isn't being shared, although for that reason you are stuck with potentially inefficient RAM allocation issues.
You skipped over paravirtualization, which can be damned fast (though not quite as snappy as LXC-based) and is theoretically in a much more secure container since the kernel is not being shared. QubesOS provides a good demonstration of how responsive a properly configured PVM can be. A "cold" boot can render a Firefox window in perhaps three seconds.
The latest version of VirtualBox supports some level of paravirtualization as well, though I haven't had time to tinker with it yet.
GPL enforcement is crucial across the board and has resulted in plenty of worthwhile code being released. OpenWRT and related firmware are great stuff that is widely used, but as I recall Linksys did not release any code until after they were threatened. These days most companies don't waste time and money fighting it, precisely because they know that it will be fought and they know they will lose.
For that reason, companies often preemptively go in the other direction and try to embrace the FOSS goodwill. Do you think Google would have dared risked shareholder lawsuits with AOSP if no one had ever bothered suing anyone for GPL violations? Do you think it's merely a staggering coincidence that Apple has made no serious effort to open source their BSD-based operating systems? (Their contributions to Darwin definitely fall under "not very serious" category.)
It's either very stupid or very disingenuous to imply that the GPL and GPL enforcement has had nothing to do with Linux's success.
(Donated money being spent on gender-specific outreach programs is another matter entirely, of course.)
Being involved in classified (and possibly off the books) Internet surveillance is not the same thing as being infallible geniuses with black van driving assassins. Recognizing that what the intelligence community has openly admitted is just the tip of the iceberg in no way means that they are utterly without incompetence... merely that not many people wish to take risks like Snowden to expose any more of the iceberg.
The NSA was not created for internet surveillance. The internet did not exist when the NSA was formed. There is no reason to think the NSA is the only three letter agency that became involved in internet surveillance and no reason to think the NRO wouldn't be interested given what we do know about them. (It's also worth musing about how both the NSA's and the NRO's existence were once classified and only revealed to the public through apparent mistakes.)
Like I said to BigFootApe, you don't need state of the art technology to authenticate or transmit sane amounts of data; you need 100 year old technology using a multi-terrabyte OTP, plus maybe a symmetric algorithm if your video feed is going to be too big for the pad size given the projected operational lifespan. It's child's play. Definitely no asymmetric encryption required, and probably no hashing either.
But I do want to say... why on Earth do so several of you seem to believe they have a super-limited scope that they never exceed? Why pretend that they are the only three letter agency that isn't at all curious about internet surveillance? Why make it a point to ignore their breathtakingly massive budget shenanigans and security policies that are pretty paranoid even by NSA or CIA standards?
No one who has worked an NRO spook job, or has talked to someone who has, would talk the way you people are talking. I'm not saying you're shills but you're acting a bit, um... naive.
Like I just explained, I'm pretty sure those are not hard problems. They were solved decades ago, in fact, and have become much easier to implement with the advent of cheap and solid state high density storage. Pretty much you just need to update the symmetric algorithm every time a better one comes along (for stuff that might be too big to encrypt via OTP, like high resolution video feeds) and nothing else needs changing. OTPs are unbreakable, blazingly fast and simple to implement and they've been around for like a hundred years.
Just because the internet security model is based on anonymous and stateless connections using least common denominator hardware doesn't mean the entire IT world operates on that principle. If the NRO spends as much on IT security as they appear to based on my own humble observations, either there's a ton of pork going on or they are doing a lot more than taking pictures with satellites. Given that pretty much every other three-letter agency has engaged in some level of internet surveillance, I think that a default assumption that the NRO's hands are clean is... naive.
If that's all they do (as some people are claiming), why the budget shenanigans and why send people to DEF CON?
There's no reason to suspect the NRO would not engage in any sort of internet surveillance. The FBI does, the NSA does, the CIA does... why would you assume that the NRO does not?
satellites are somehow useful in tapping fiber optic cables.
Yes. They have nothing but satellites at NRO facilities. Satellites and satellite dishes. No computers or telecommunications equipment whatsoever. The eggheads deep in their secret underground bases haven't even heard of fibre optics yet. Our intelligence community is rigidly segmented with absolutely no overlapping activities or duties whatsoever, conducted in complete transparency and cooperation, definitely with no dark projects whatsoever being funded by $1.5+ billion that the CIA noticed the NRO was very quietly sitting on.
You could just as well argue that no one who works for the NRO or NSA carries a gun, because only the FBI is allowed to do that. You're wrong both in principle and in practice.
Who doesn't? It is the largest infosec conference, not a very informative one, but the networking opportunity is worth every cent.
Nutria said the NRO was "just satellites" and wasn't involved in internet surveillance. Nobody who was doing "just satellites" would have that much use for infosec skills, even taking into account the need for secure authentication and encryption. (That may be a huge problem on the internet, but everything gets simpler and much more robust when both sides can be stateful with preshared secrets. You can use OTPs to inject noise into any cryptographic process.)
It depends on what your motivation for using TOR is. The encryption and obfuscation works in both directions, so the ISP and/or whoever owns the access point you're using (coffee shop, employer, etc.) and/or honeypot operators are not able to spy on or hijack your Facebook session.
TOR may be overkill for that use case, but it's free and arguably easier to use vs. a commercial VPN.
That said, a TOR exit node is going much more suspect then your average coffee shop access point.
Slashdot Mirror
HVM is not is not faster than PVM. You might be confusing PVM for the software-driven virtualization that VMWare/QEMU/Virtualbox/etc. used prior to the introduction of VT-x.
From my understanding, paravirtualization works by running a modified kernel in the guest so that knows it's really in userspace of the host OS (Ring 3) instead of where it would normally live (Ring 0). VT-x basically adds additional namespaces that allow guest operating systems to run in Ring 0, but this process adds some overhead. And for some reason I've noticed this leads to some massive slowdowns in some applications, especially anything that does a lot of FLOPS.
I'm not an expert in these matters, but everything I've seen says (and my own experience verifies) that paravirtualization is generally faster than HVM. I'm running Qubes right now and all of my paravirtualized applications respond normally. Guest OSes in VirtualBox with VT-x enabled (on this same exact hardware) could get quite sluggish.
I've never used Docker but it is (so I gather) geared largely towards LXC virtualization. This is radically different from either PVM or HVM as it is basically creating multiple namespaces within a single kernel. It should be faster than either HVM or PVM, though still not as fast as a chroot. Depending on your goals, I don't necessarily agree that it is "best suited for application-oriented virtualization." It's fast and it removes the problem of RAM management, but from a security standpoint the compartmentalization isn't nearly as airtight and you lose the flexibility of being able to utilize different kernels. The overhead of the alternatives can be non-trivial, but silicon is cheap. Like I said, it depends on your goals.
All of our commonly used, general purpose, end user operating systems suck. Separation, compartmentalization simply isn't there. Everyone knows the advantages of proper compartmentalization: easy portability, unparalleled security, a mighty robustness against system-crashing errors and reboots, configuration flexibility and avoidance of dependency conflicts, the ability to easily prioritize system resource usage, the ability to roll back snapshots without rebooting the entire system, sharing or cloning base images, etc. But our OSes can't do any of this out of the box.
So people slowly added it back in, first with file permissions and then mandatory access control (Apparmor, SELinux), chroots, FreeBSD chroots, traditional virtualization (at first entirely software-based, then hardware-assisted), PVMs, LXC, Unikernel, and probably a dozen more approaches I'm unaware of. All of these things have advantages and disadvantages, but the important thing to keep in mind is this: they're all trying to fix something that is fundamentally broken. And ultimately all of the best solutions involve figuring out different ways of emulating multiple OSes because the applications we want to run only run on "modern" OSes... none of which are built around a concept of strict compartmentalization.
Enter the Genode aficionados and the hordes of older microkernel partisans who've been screaming about this stuff for decades. They will scoff and say a bunch of undeniably true things... but of course, last time I checked everyone who actually used Genode for everyday tasks was using it to boot VirtualBox or something similar. It's inescapable: we have a bunch existing applications we want to run and we simply can't run it without (in some fashion) running the crappy OS it was designed to run on.
And so we have a dozen different varieties of duct tape to choose from, each with their own advantages and disadvantages. It's interesting. It's fascinating. Currently, I'm leaning towards paravirtalization as the sweet spot of performance and flexibility and security (Qubes OS was the decisive influence on me here), but LXC is faster and simpler and Unikernel, while not simple, sounds like it could be very fast indeed. And I'm sure the dirt-simple appeal of vanilla HVMs will endure regardless of the advances Docker makes.
It's an interesting debate with no single right answer because of the long list of different pros and cons. But it's also interesting to consider how and when all of this is likely to be rendered moot. At some point, a new OS (probably microkernel based) will finally gain enough traction and get enough native end user applications to make it usable without in any fashion having to emulate another OS on top of it. People will start porting other applications to run on it, and eventually the Windows and Debian and Red Hat and OS X and Android and iOS of today will finally be abandoned as the insecure, unstable, kludge-ridden clusterf___s they are.
The question is... will this revolution be driven by an OSS project? Will the microkernel world finally get their Linus? Or will this be the ultimate proprietary walled garden, a massive investment by Apple or Microsoft or Google, some heart-stoppingly expensive super secret project to finally offer their customers a UI that never gets laggy or unresponsive, a core system that is virtually immune to malware, app developer APIs that rarely need changing, real time performance when necessary, and fine-grained app permissions that actually work properly?
Unfortunately, I think my money has to be on the walled garden, and even more unfortunately I'd have to say that Apple stands the best chance of pulling it off given their strict control over hardware and their titanic cashflows. Maybe Google will get there first and maybe, just maybe someone will be able to convince them to open source it.
It's fascinating to watch all of these brilliant people come up with these interesting and inventive kludges. And useful, undeniably useful in the here and now. But at the end of the day, I can't help but think... dust in the wind.
The layers add abstraction, compartmentalization, portability and the ability to roll back snapshots without rebooting. If you understand virtualization then I'm sure you can understand how this can greatly increase security and robustness in general, at least in principle.
Now, if you don't understand why someone would use an LXC-based solution like Docker instead of a fully virtualized HVM machine then you probably haven't seen them in practice. They are native speed, and you don't have to mess around with inefficient memory allocation. The performance difference is critical in many cases and even when it isn't, it's still damn nice to have everything being as smooth as normal. Unikernel is a different approach, but offers similar performance advantages over traditional HVMs.
As I mention below, paravirtualization is also pretty damn fast and is more secure and flexible compared to LXC solutions since the kernel isn't being shared, although for that reason you are stuck with potentially inefficient RAM allocation issues.
You skipped over paravirtualization, which can be damned fast (though not quite as snappy as LXC-based) and is theoretically in a much more secure container since the kernel is not being shared. QubesOS provides a good demonstration of how responsive a properly configured PVM can be. A "cold" boot can render a Firefox window in perhaps three seconds.
The latest version of VirtualBox supports some level of paravirtualization as well, though I haven't had time to tinker with it yet.
GPL enforcement is crucial across the board and has resulted in plenty of worthwhile code being released. OpenWRT and related firmware are great stuff that is widely used, but as I recall Linksys did not release any code until after they were threatened. These days most companies don't waste time and money fighting it, precisely because they know that it will be fought and they know they will lose.
For that reason, companies often preemptively go in the other direction and try to embrace the FOSS goodwill. Do you think Google would have dared risked shareholder lawsuits with AOSP if no one had ever bothered suing anyone for GPL violations? Do you think it's merely a staggering coincidence that Apple has made no serious effort to open source their BSD-based operating systems? (Their contributions to Darwin definitely fall under "not very serious" category.)
It's either very stupid or very disingenuous to imply that the GPL and GPL enforcement has had nothing to do with Linux's success.
(Donated money being spent on gender-specific outreach programs is another matter entirely, of course.)
Being involved in classified (and possibly off the books) Internet surveillance is not the same thing as being infallible geniuses with black van driving assassins. Recognizing that what the intelligence community has openly admitted is just the tip of the iceberg in no way means that they are utterly without incompetence... merely that not many people wish to take risks like Snowden to expose any more of the iceberg.
The NSA was not created for internet surveillance. The internet did not exist when the NSA was formed. There is no reason to think the NSA is the only three letter agency that became involved in internet surveillance and no reason to think the NRO wouldn't be interested given what we do know about them. (It's also worth musing about how both the NSA's and the NRO's existence were once classified and only revealed to the public through apparent mistakes.)
https://en.wikipedia.org/wiki/...
I suppose there's technically supposed to be a space there. My mistake. I can see how the ambiguity would be very troubling.
Like I said to BigFootApe, you don't need state of the art technology to authenticate or transmit sane amounts of data; you need 100 year old technology using a multi-terrabyte OTP, plus maybe a symmetric algorithm if your video feed is going to be too big for the pad size given the projected operational lifespan. It's child's play. Definitely no asymmetric encryption required, and probably no hashing either.
But I do want to say... why on Earth do so several of you seem to believe they have a super-limited scope that they never exceed? Why pretend that they are the only three letter agency that isn't at all curious about internet surveillance? Why make it a point to ignore their breathtakingly massive budget shenanigans and security policies that are pretty paranoid even by NSA or CIA standards?
No one who has worked an NRO spook job, or has talked to someone who has, would talk the way you people are talking. I'm not saying you're shills but you're acting a bit, um... naive.
Like I just explained, I'm pretty sure those are not hard problems. They were solved decades ago, in fact, and have become much easier to implement with the advent of cheap and solid state high density storage. Pretty much you just need to update the symmetric algorithm every time a better one comes along (for stuff that might be too big to encrypt via OTP, like high resolution video feeds) and nothing else needs changing. OTPs are unbreakable, blazingly fast and simple to implement and they've been around for like a hundred years.
Just because the internet security model is based on anonymous and stateless connections using least common denominator hardware doesn't mean the entire IT world operates on that principle. If the NRO spends as much on IT security as they appear to based on my own humble observations, either there's a ton of pork going on or they are doing a lot more than taking pictures with satellites. Given that pretty much every other three-letter agency has engaged in some level of internet surveillance, I think that a default assumption that the NRO's hands are clean is... naive.
If that's all they do (as some people are claiming), why the budget shenanigans and why send people to DEF CON?
There's no reason to suspect the NRO would not engage in any sort of internet surveillance. The FBI does, the NSA does, the CIA does... why would you assume that the NRO does not?
It's adorable how people think that ...
satellites are somehow useful in tapping fiber optic cables.
Yes. They have nothing but satellites at NRO facilities. Satellites and satellite dishes. No computers or telecommunications equipment whatsoever. The eggheads deep in their secret underground bases haven't even heard of fibre optics yet. Our intelligence community is rigidly segmented with absolutely no overlapping activities or duties whatsoever, conducted in complete transparency and cooperation, definitely with no dark projects whatsoever being funded by $1.5+ billion that the CIA noticed the NRO was very quietly sitting on.
You could just as well argue that no one who works for the NRO or NSA carries a gun, because only the FBI is allowed to do that. You're wrong both in principle and in practice.
Oh yeah, and the NRO sends people to DEFCON.
Who doesn't? It is the largest infosec conference, not a very informative one, but the networking opportunity is worth every cent.
Nutria said the NRO was "just satellites" and wasn't involved in internet surveillance. Nobody who was doing "just satellites" would have that much use for infosec skills, even taking into account the need for secure authentication and encryption. (That may be a huge problem on the internet, but everything gets simpler and much more robust when both sides can be stateful with preshared secrets. You can use OTPs to inject noise into any cryptographic process.)
You guess wrong. (That's the NSA. The NRO is just satellites.)
It's adorable how people think that only one agency is involved with mass internet & telecom surveillance.
The NRO has a budget comparable to the NSA and they have already been caught hiding billions of dollars of their budget for undeclared purposes.
Oh yeah, and the NRO sends people to DEFCON.
It depends on what your motivation for using TOR is. The encryption and obfuscation works in both directions, so the ISP and/or whoever owns the access point you're using (coffee shop, employer, etc.) and/or honeypot operators are not able to spy on or hijack your Facebook session.
TOR may be overkill for that use case, but it's free and arguably easier to use vs. a commercial VPN.
That said, a TOR exit node is going much more suspect then your average coffee shop access point.