When people talk about Blue Pill as being "undetectable" they mean "through the use of a program."
And that's Joanna's point. Properly constructed, Blue Pill 2 (the successor with full emulation support coded in--she herself admitted that her prototype is imperfect) would be undetectable by software running inside the VM.
This is the fundamental problem I have. So she has a crappy prototype but claims that the next version will be undetectable? Where's the paper? What is she exploiting to make this actually work?
She's got a theoritically "undetectable" exploit for which there is a theoritical way to detect it. Doesn't that seem a little odd? How big do you think Blue Pill 2 would have to be? Just to make the VMM itself would require something akin to Xen or VMware. We're talking hundreds of thousands of lines of code. Is that really a practical exploit in large enterprises?
Even with a full emulator, you cannot keep the VM from consulting external time sources in general. Just fixing up the TSC is not enough.
This is hardware virtualization we're talking about, not software. The processor manufacturers have built virtualization calls into their chipsets.
They've added extensions to facilitate trap-and-emulate virtualization.
The side-effect of this is that the hypervisor can simply tell the bios "I'm the hypervisor...but, only call me when these specific requests are made."
VT/SVM have absolutely nothing to do with the BIOS. Instead, they both introduce a new processor mode that can be entered at any point that allow certain operations to be trapped. These operations are more or less the set of classic x86-sensitive instructions.
So, the hypervisor could simply choose to ignore the sound and video hardware, leaving those as fast as they were before.
Yup. But we're not talking about hardware here. Keep in mind, that if you do allow direct access to hardware, one now has a channel to access all of memory which could be used to detect a virus in RAM. Let's ignore that for now though:-)
The only way to tell the hypervisor is there is to find a CPU call that the hypervisor *does* care about, and compare how long it takes to run that command before & after the rootkit pushes the OS to a guest OS.
Yup, and as I pointed out to Joanna, there are a number of CPU operations that one would *have* to trap. Things like %cr3 moves, msr reads/writes, etc. Otherwise, one can just search memory for a signature. BTW, how does she hide the memory of the VMM from the guest? I didn't address this because there are some potential solutions (like memory hotplug) but this, in practice, would be a very hard problem to solve. You can just take away memory from an Operating System and expect things to function normally.
Why do you think she addressed this in her talk? I brought it up to her before she presented...
Anyway, you have to take a trap at some point. There are only a small number of possible instructions that trap. A very thorough "detector" could simply check the timing of every trapable instruction.
If she's not trapping any instructions, then the monitor is never getting run so is it really a monitor anymore?
BTW, on VT at least, the VMM doesn't get a choice for certain instructions. They always trap no matter what.
Not the only one to come to this conclusion...
on
Blue Pill Myth Debunked
·
· Score: 2, Informative
If mosquito and similar tools are not moving towards VMMs, I'd be very suprised. After all, it is a logical step (From VM as a payload, to a VMM as a payload).
Of course, VMM's can be used to do all sorts of nasty things. VMM-level virus could certainly be nasty. And, an important point to note, is that it may be entirely possible for a virus to be hidden in a VMM and for a virtual machine not to be able to detect that virus. Will VMM's need anti-virus software? I hope they don't suck that much.
What "blue pill" is though is something much different. It's claim is that you can take a native Operating System and turn it into a virtual machine without the OS knowing about it.
Never say impossible when logic says it could be done. Just say impractical..
There are actually things in computer science that are impossible. Usually, they are problems in the form "figure out whether another program has propery X". Classic examples are the halting problem.
Recall, I'm disputing the claim of "100% undetectable". You could make something that's really, really, really hard to detect.
A lot of realtime code is run along side the kernel under a rudimentary hypervisor (Google for nanokernels, Adeos and RTLinux do this sort of thing). In this very important case, it is usually quite a pain to require the OS to have to implement the infrastructure to support emulated devices when it could be using a hypercall infrastructure like Xen.
You clearly haven't ever written a driver for Xen. The Xen drivers are far more complicated than normal hardware. This is mostly do to XenBus but that's a whole other conversation....
However, in addition to this, Xen caters to situations with tiny realtime handlers running along side the a larger interface OS.
What do you base this claim on? I can write a very tiny OS that runs with emulated hardware is considerably less space than what it takes to run under Xen.
If you desire to have any real understanding of the issues, take a look at the VMI spec and then the Xen Hypercall docs. Note the proliferation of x86 instructions and constructs in the former and the clean implementation of abstract interfaces in the latter.
Uh, you do realize that there are more Xen hypercalls than VMI exits? There are some VMI exits that aren't strictly needed (because they are ops that can be trapped and emulated) but the reasoning behind including these in VMI was so that one could write a very minimialistic hypervisor that did no instruction emulation (doing emulation correctly is *very* complicated).
You also realize that the Xen hypercall API varies greatly across architectures right?
So you're way off base on all your assertions. The real history of Xen is that it is based on the Nemesis exokernel which was a research project out of the University of Cambridge. Long before everyone was excited about hypervisors, a number of research exokernernels/microkernels were capable of running Linux as a guest (L4, K42, and Nemesis). This "high level" interfaces that you're referring to are actually part of the nemesis API (event channels, domains, etc--although grant tables is a new concept that was pretty recently introduced).
Now, Xen paravirtual is different from other forms (like what appears in Denali or even PowerPC) because it makes the guest aware of these higher level APIs. The problem is that this makes maintaince a nightmare because the changes to the guest are so extreme.
So, after two years of this guess what's happening? Xen is ditching writable page tables in favor of shadow paging and adopting Rusty Russell's paravirt_op architecture. paravirt_op is very similar to VMI (it's actually a subset right now). The idea is that this isolates the complex Xen API from the guest to reduce the mainaince burdern (which is way too high right now). There are going to be cases where details of the Xen API must escape the current paravirt_ops but we're going to examine these on a case-by-case basis and try to reduce that as much as possible. Contrast that to the former approach where there was basically no attempt to isolate the Xen API from the guest.
The problem with VMI is not the fact that the API is too low-level. The real problems center around whether a binary blob should be allowed to be injected into the kernel by the hypervisor and the desire of the kernel community to have a stronger relationship between the kernel and the hypervisor.
Hah! I knew it. Xen's just not going for the VMI approach because it's pretty much what L4 guys have supported for quite some time. You couldn't help but do some "friendly" bashing, could you.;-) (Oh well, neither could I.)
No, L4 hasn't supported it for a long time--only just recently. I said they don't count because they're doing it with previrtualization which is exceedingly cooler than any of this other stuff.
Performance in full virtualized mode is noticeably worse than in paravirtual mode, so people are fighting over paravirtual interface standards.
For now, this is true. The expectation is that over time, using hardware virtualization will be beneficial. The paravirt_ops interface is designed so that there's at least one op corresponding to every trappable instruction. In the future, we'll be able to rewrite the calling location of a paravirt_op. One of the reasons for this is so that we can make the decision to use the normal instructions when the hardware catches up.
FWIW, I suspect that SVM is probably already pretty good because of the tagged TLB.
One that was set on HARDWARE instead, a virtualization support at processor level?
No, that's not what this discussion is about. This is about a *higher* level interface than that provided by hardware. This whole debate is really about an API of less than 10 functions that do higher level operations. By hooking this higher level operations into the hypervisor, you can dramatically improve performance.
A canonical example is updating page tables. Normally, if you were to hook at the hardware level, updating a page table on a fork may require, let's say, 1,000 interactions with the hypervisor. By making a small change to the guest (paravirtualization), this operation may only take one interaction.
Paravirtualization is still useful in the presence of hardware virtualization which is why this debate is so important. It's very relevant to future hardware.
Disclaimer: Xen hacker, personal opinion:-) Didn't think it was necessary to mention but what the heck:-)
VMware went to OLS and presented a paper demonstrating a VMI interface that runs either Xen or VMware at the same speed as the Xen interface.
I missed OLS unfortunately but I've come to the conclusion that VMI is not the best interface for Xen long term. I should say that I was advocating VMI for a long time previously so this isn't just a knee-jerk reaction.
The problem with VMI is that it hides the hypervisor interface from the guest. Now, if you're VMware, this is ideal. If you're a Free Software project, and your interfaces are evolving overtime, having your interfaces hidden means that there's no pressure to improve them.
The biggest benefit for upstream merge for Xen will be a ton of hackers on LKML auditing the interfaces and saying where they suck and forcing us to change them. You don't want to hide the virtual timer interface behind PIT emulation. You want codesign.
This doesn't mean that Linux shouldn't support VMI. It just means that not all projects should standardize on the VMI ABI.
Xen has never tried to run on a VMware hypervisor, but XenSource went and signed a deal to run on the (future) Windows hypervisor. My opinion is that Xen is a bunch of hypocrites: they complain about how VMware isn't open, then go sign a deal with the least open company of all. Of course, I'm biased.
Please try to separate XenSource from the Xen community. Many of us don't work for XenSource and many of us think that XenSource does stupid things (this being a good example).
Xen wants VMware to adopt the Xen hypervisor interface.
Many of us don't even want Xen to use its own interface. Why would we wish it upon VMware:-) The problem with a VC-funded company associated with Xen is that some brand new executive makes some silly statement and it's treated as the official opinion of the Xen community.
Imagine if every exec at every company loosely associated with Linux was quoted as gospel for Linux's future.
This is impossible. The Xen interface is too tightly coupled to the Xen hypervisor; it's missing pieces that are necessary to run the VMware hypervisor at reasonable performance. VMware doesn't really care which interface actually proliferates (as in, there will be a layer of interface glue regardless), so long as the interface is good enough. Xen's interface is not good enough. As of two weeks ago, Xen and VMI were the only two interfaces out there.
This is absolutely correct. The Xen interface is not rich enough to support a variety of hypervisors with reasonable performance. Anyone who claims differently is lying to you:-)
Greg K-H's gripe with VMware is that the kernel module isn't open source. Yes and no (I don't want to argue - the code is open but not GPLed), the point is that he's spending more time complaining about Xen and VMware than it would take to actually mediate the problem.
Which is a valid gripe. VMware is going to get the short-end of the stick when interacting with the kernel community because they are doing something that is viewed both as immoral and illegal. There's an easy way to fix that...
Honestly, if there was a single Free hypervisor that worked with VMI (and L4 doesn't count;-)), VMI would already be in the kernel.
(Which, thankfully, someone else is doing instead, with paravirt_ops).
Yes, and this is going to be a very long process. I will say too that engineers from Xen and VMware have both been working together surprisingly well on this. There is an active conversation going on in the osdl-virtualization list and on other channels. Despite these stories, Xen and VMware are actually working together.
Finally: I saw more pot-shots about being unable to benc
Subversion only, no real mailing list integration, no real web hosting to speak of. The lack of ads is nice but what I really want is more flexible revision control (I'd like to use mercurial), mail man integration (that doesn't lag like crazy like sf), and a place that I can host a wiki.
The FAQ on the XenSource site make it pretty clear that the result of this partnership will be a commercial product. Specifically, one that does hypercall translation from the Xen ABI to the Microsoft Hypervisor ABI.
The Xen ABI is based on a "hypercall page". This is essentially a table of function stubs. The default hypercall page just does hypercalls for each stub. I presume that XenSource is building a special plugin for the Windows hypervisor that offers a Xen hypercall page to the guest. That page would do hypercall translation to the Microsoft Hypervisor ABI. This would require the ability to write a plugin the the Microsoft hypervisor (hence the cooperation) but it pretty much independent of Xen.
Of course, this leads to an interesting question. Since the hypervisor page runs in the same ring (and address space) as the Linux kernel, wouldn't it have to be GPL'd? Certainly, if you agree that binary modules violate the GPL, this can be viewed as nothing but a special purpose kernel module (and therefore a GPL violation).
It's smart that Microsoft is partnering here to limit it's own legal liability.
FWIW, I am not a lawyer and am purely speculating based only on the publicly available information on the XenSource site.
A key point about virtualization (even hardware virtualization) that people miss is that it does not guarentee that programs run as they normally if those programs are timing sensitive. This isn't a new revelation. If you go back to the Popek/Goldberg paper from the 70s, they make it quite clear.
So how do you exploit this to detect that you're in a VM? If you're an operating system, the easiest approach is to disable interrupts periodically and wait out a few time slices. You would then compare wallclock time and see if you're wait took longer than you expected it should. If it did, you're being pre-empted. With interrupts off, that's a sure sign that you're in a VM.
The above is a general solution to the problem. It's funny the author used SVM (a.k.a. Pacifica). SVM has a feature called dynamic attestation. This essentially introduces an unemulatable instruction that one can use precisely for the purpose of determining whether you're in a VM or not.
Python is not something that should be used to write future OS code either.
I think in this case something like Java syntax makes more sense.
I find it interesting that you refer to Java as a high level language. Okay, let's run with it though. You cannot have this problem in Java exactly because functions are not first class types. Java is a very restricted language in this respect. Of course, it's not entirely type safe. Consider:
class A { public int foo() { return 0; } } class B extends A { } class C extends A { }
public class test {
public static void main(String args[]) {
A a[] = new B[20];
a[0] = new C();
System.out.println(a[0].foo());
} }
This gem will compile with no compile time error (even though it clearly breaks the type system). Java arrays are quite silly semantically. A good example of a really strictly typed language is either ML or Haskell.
The problem (if there is one) lies with the language, not the compiler, since both of the above lines are legal C code. Solutions to this kind of problem probably involve both a movement towards higher level languages (which are typically more verbose and don't allow low-level memory manipulation)
I think we can both agree Python is a higher level language. And guess what:
import os
if os.getuid() != 0 or os.geteuid = 0:
is completely valid. It's not high level vs low level languages here that's at issue. It's static verses dynamic typing and more specifically, strict verses weak static typing. If 0 wasn't treated so specially in C (it's the only numeric literal that's directly comparable to a pointer) this wouldn't be an issue.
Unfortunately, C++ made it even worse since the standard mandates that NULL is defined as:
No, they don't. Market driven is boring. People don't work late at night in their free time to do boring things. Successful Open Source projects are all about having fun. Open Source developers are weird and quirky. The only people who care about Open Source being "market driven" are people who expect Open Source to be exactly like commercial software but free. That's not what it is and that's never what it will be.
Something seems fishy about this, though that may be because it's 6:30 in the morning as I begin to write this./me waves hands
Well, it's sort of fishy as 100 choose 2 returns pairs that may, for instance, be (a, b) and (b, a). You could remove duplicates by dividing by two but it's not going to change very much in the resulting answer. The key point to take away is that it's a very very large number:-)
At first glance, I'm not sure HOW this is an "unsolvable" problem. Would I not just select and group 100 students at random then rearrange the pairs as I found incompatibilities? Can someone clue me in to what I'm missing here?
What makes a problem NP is not whether it's solvable but rather how long it takes to solve. The algorithm you propose is a search algorithm. Consider what would happen if your list of incompatible students was so large that within the group of 100 students you randomly choose, there is not a single possible arrangement of pairs. This means you would have to choose another group of 100 students. It's a minor refinement but an important one.
Now consider if that list was so large that there was only a single possible group of 100 that contains an arrangement of pairs that worked. Now consider that within that group of 100, there was only one good possible arrangement. If you're very unlucky, and you choose these set of 100 and arrangement of pairs last, you have to try every possible combination before finding the right one. Okay, so what?
Lets see how many possible answers you'd have to try. Within a group of 100 students, there are 100 choose 2 possible arrangements. There are 400 choose 100 possible choices of 100 students. n choose k is really n! / (k! (n-k)!) where n! is n * (n - 1) *... * 1. Since we're trying every possible combination, this gives us:
[400! / (100! 300!)] * [100! / (2! 98!)]
Your standard calculator is not going to be able to solve this one but if you have an arbitrary precision calculator (like bc), you get:
Which is an awfully large number. That number is so large, in fact, that even if you have a computer that could check one possible solution with every electron in the universe, until the Sun supernova's, you'd still not find the answer.
Now, that depends on really bad luck. You can construct problems though that given average luck, you would not find the solution in the lifetime of the universe. This is what cryptography is based on.
Compare this to a standard sorting algorithm. To sort the list [3, 4, 5, 6, 7, 8, 9, 2, 1, 0] given a crappy algorithm like bubble sort requires n*n = 100 computations. You can solve this problem the same way using search though. You merely have to randomly arrange the list in every possible way and check to see if your random arrangement is sorted. There are n! possible arrangements of a list of n elements so there are 10! = 3628800 possible answers to search. You can see that even a crappy algorithm like bubble sort is much better than search.
The difference is even greater with larger lists. A problem that is only solvable via search is considered NP. A problem that is solvable with an algorithm in polynomial time (n*n is a polynomial) is considered P. The N in NP stands for non-polynomial.
So the problem here is whether there exists a polynomial solution for these set of problems that we've labelled NP. What makes this even more significant is that it has been proven that if we find a polynomial solution for one NP problem, we can create solutions for any NP problem. A lot is riding on the lack of existence of a polynomail solution for NP problems. If someone where to prove that there are indeed polynomial solutions to NP problems it would be earth-shattering. After the initial shock, it would also open up a whole new world of mathematics since a lot of things we didn't think were possible to do efficiently became possible.
I noticed that whenever virtualization comes up, no one ever mentions CoLinux.
That's because no one really takes it seriously. It's not that it's not novel or not interesting, but the Linux guest runs as a *kernel thread*, not as a process. This means the guest has as much access to hardware as the host. It relies on a very well behaved Linux guest to not bring your system to a screaming halt.
If CoLinux ever adapts to run the Linux guest in a lesser ring (perhaps 1 or 2) then it will be considerably more interesting.
OK guys, now I'm confused. WTF is going on here? Have the Xen people been bought my Microsoft?
No, TFA is just misleading. A while back, Microsoft documented the VHD format under an anti-GPL license (similar to what they did with CIFS a while ago). XenSource, the commercial company trying to make money from Xen, simply downloaded this license after agreeing to the terms and announced that their product, XenEnterprise, will support importing VHD hard disks.
The preview feature is useless because 1) it's not on every page that comes back and 2) it's not updated live.
1) Just because it doesn't come for every site doesn't make it useless. It clearly has a use for pages that it works on. It may be less useful but it not useless.
2) The fact that it's not updated live does not make it useless either. It is quite useful, for instance, in quickly determining if a site is a link-farm or is going to load some awful flash piece of crap.
Esclation of Privileges is a vulnerability, the last time I checked.
According to whom? If it isn't, why do so many programs go to such great trouble to reduce privilege level when running? Hell, why do we even have users in the first place?
But they are in ogg format. Didn't even know there was a ogg video format. As long as the Linux world continues to alienate those using Windows, I don't care about it.
Then disable the Linux section in your slashdot main page and stop complaining.
Why couldn't they just standardize on Xgl? It works *today*. Aigl doesn't even support my nvidia card right now.
It doesn't Just Work. The integration with Gnome is very poor (you can do awful things like hide window bars under the panel--doh!). It uses a whole new Window Manager (WM's are tricky to get right so quite a few apps don't work now). emacs doesn't start for me under Ubuntu (the xrdb seems to be screwed up).
Plus, Xgl for ATI cards is very buggy (due to the ATI drivers) and require running Xgl on:1. Plus, some things are noticably choppier (web browswer scrolling).
Some of the effects are cool in Xgl (the window scaling for instance) but some are annoying (like the new switcher). Being able to shut of GL effects *without* restarting X is a big deal. If I'm just trying to program, I don't want wobbly windows or any of that stuff.
Slashdot Mirror
When people talk about Blue Pill as being "undetectable" they mean "through the use of a program."
And that's Joanna's point. Properly constructed, Blue Pill 2 (the successor with full emulation support coded in--she herself admitted that her prototype is imperfect) would be undetectable by software running inside the VM.
This is the fundamental problem I have. So she has a crappy prototype but claims that the next version will be undetectable? Where's the paper? What is she exploiting to make this actually work?
She's got a theoritically "undetectable" exploit for which there is a theoritical way to detect it. Doesn't that seem a little odd? How big do you think Blue Pill 2 would have to be? Just to make the VMM itself would require something akin to Xen or VMware. We're talking hundreds of thousands of lines of code. Is that really a practical exploit in large enterprises?
Even with a full emulator, you cannot keep the VM from consulting external time sources in general. Just fixing up the TSC is not enough.
This is hardware virtualization we're talking about, not software. The processor manufacturers have built virtualization calls into their chipsets.
:-)
They've added extensions to facilitate trap-and-emulate virtualization.
The side-effect of this is that the hypervisor can simply tell the bios "I'm the hypervisor...but, only call me when these specific requests are made."
VT/SVM have absolutely nothing to do with the BIOS. Instead, they both introduce a new processor mode that can be entered at any point that allow certain operations to be trapped. These operations are more or less the set of classic x86-sensitive instructions.
So, the hypervisor could simply choose to ignore the sound and video hardware, leaving those as fast as they were before.
Yup. But we're not talking about hardware here. Keep in mind, that if you do allow direct access to hardware, one now has a channel to access all of memory which could be used to detect a virus in RAM. Let's ignore that for now though
The only way to tell the hypervisor is there is to find a CPU call that the hypervisor *does* care about, and compare how long it takes to run that command before & after the rootkit pushes the OS to a guest OS.
Yup, and as I pointed out to Joanna, there are a number of CPU operations that one would *have* to trap. Things like %cr3 moves, msr reads/writes, etc. Otherwise, one can just search memory for a signature. BTW, how does she hide the memory of the VMM from the guest? I didn't address this because there are some potential solutions (like memory hotplug) but this, in practice, would be a very hard problem to solve. You can just take away memory from an Operating System and expect things to function normally.
Why do you think she addressed this in her talk? I brought it up to her before she presented...
Anyway, you have to take a trap at some point. There are only a small number of possible instructions that trap. A very thorough "detector" could simply check the timing of every trapable instruction.
If she's not trapping any instructions, then the monitor is never getting run so is it really a monitor anymore?
BTW, on VT at least, the VMM doesn't get a choice for certain instructions. They always trap no matter what.
FWIW, Keith Adams of VMware posted a recent blog entry "Blue Pill" is quasi-illiterate gibberish and there have been a number of other folks that have come to the same conclusion.
If mosquito and similar tools are not moving towards VMMs, I'd be very suprised. After all, it is a logical step (From VM as a payload, to a VMM as a payload).
Of course, VMM's can be used to do all sorts of nasty things. VMM-level virus could certainly be nasty. And, an important point to note, is that it may be entirely possible for a virus to be hidden in a VMM and for a virtual machine not to be able to detect that virus. Will VMM's need anti-virus software? I hope they don't suck that much.
What "blue pill" is though is something much different. It's claim is that you can take a native Operating System and turn it into a virtual machine without the OS knowing about it.
Never say impossible when logic says it could be done. Just say impractical..
There are actually things in computer science that are impossible. Usually, they are problems in the form "figure out whether another program has propery X". Classic examples are the halting problem.
Recall, I'm disputing the claim of "100% undetectable". You could make something that's really, really, really hard to detect.
A lot of realtime code is run along side the kernel under a rudimentary hypervisor (Google for nanokernels, Adeos and RTLinux do this sort of thing). In this very important case, it is usually quite a pain to require the OS to have to implement the infrastructure to support emulated devices when it could be using a hypercall infrastructure like Xen.
You clearly haven't ever written a driver for Xen. The Xen drivers are far more complicated than normal hardware. This is mostly do to XenBus but that's a whole other conversation....
However, in addition to this, Xen caters to situations with tiny realtime handlers running along side the a larger interface OS.
What do you base this claim on? I can write a very tiny OS that runs with emulated hardware is considerably less space than what it takes to run under Xen.
If you desire to have any real understanding of the issues, take a look at the VMI spec and then the Xen Hypercall docs. Note the proliferation of x86 instructions and constructs in the former and the clean implementation of abstract interfaces in the latter.
Uh, you do realize that there are more Xen hypercalls than VMI exits? There are some VMI exits that aren't strictly needed (because they are ops that can be trapped and emulated) but the reasoning behind including these in VMI was so that one could write a very minimialistic hypervisor that did no instruction emulation (doing emulation correctly is *very* complicated).
You also realize that the Xen hypercall API varies greatly across architectures right?
So you're way off base on all your assertions. The real history of Xen is that it is based on the Nemesis exokernel which was a research project out of the University of Cambridge. Long before everyone was excited about hypervisors, a number of research exokernernels/microkernels were capable of running Linux as a guest (L4, K42, and Nemesis). This "high level" interfaces that you're referring to are actually part of the nemesis API (event channels, domains, etc--although grant tables is a new concept that was pretty recently introduced).
Now, Xen paravirtual is different from other forms (like what appears in Denali or even PowerPC) because it makes the guest aware of these higher level APIs. The problem is that this makes maintaince a nightmare because the changes to the guest are so extreme.
So, after two years of this guess what's happening? Xen is ditching writable page tables in favor of shadow paging and adopting Rusty Russell's paravirt_op architecture. paravirt_op is very similar to VMI (it's actually a subset right now). The idea is that this isolates the complex Xen API from the guest to reduce the mainaince burdern (which is way too high right now). There are going to be cases where details of the Xen API must escape the current paravirt_ops but we're going to examine these on a case-by-case basis and try to reduce that as much as possible. Contrast that to the former approach where there was basically no attempt to isolate the Xen API from the guest.
The problem with VMI is not the fact that the API is too low-level. The real problems center around whether a binary blob should be allowed to be injected into the kernel by the hypervisor and the desire of the kernel community to have a stronger relationship between the kernel and the hypervisor.
Hah! I knew it. Xen's just not going for the VMI approach because it's pretty much what L4 guys have supported for quite some time. You couldn't help but do some "friendly" bashing, could you. ;-) (Oh well, neither could I.)
No, L4 hasn't supported it for a long time--only just recently. I said they don't count because they're doing it with previrtualization which is exceedingly cooler than any of this other stuff.
Performance in full virtualized mode is noticeably worse than in paravirtual mode, so people are fighting over paravirtual interface standards.
For now, this is true. The expectation is that over time, using hardware virtualization will be beneficial. The paravirt_ops interface is designed so that there's at least one op corresponding to every trappable instruction. In the future, we'll be able to rewrite the calling location of a paravirt_op. One of the reasons for this is so that we can make the decision to use the normal instructions when the hardware catches up.
FWIW, I suspect that SVM is probably already pretty good because of the tagged TLB.
One that was set on HARDWARE instead, a virtualization support at processor level?
No, that's not what this discussion is about. This is about a *higher* level interface than that provided by hardware. This whole debate is really about an API of less than 10 functions that do higher level operations. By hooking this higher level operations into the hypervisor, you can dramatically improve performance.
A canonical example is updating page tables. Normally, if you were to hook at the hardware level, updating a page table on a fork may require, let's say, 1,000 interactions with the hypervisor. By making a small change to the guest (paravirtualization), this operation may only take one interaction.
Paravirtualization is still useful in the presence of hardware virtualization which is why this debate is so important. It's very relevant to future hardware.
Disclaimer: VMware employee, personal opinion.
:-) Didn't think it was necessary to mention but what the heck :-)
:-) The problem with a VC-funded company associated with Xen is that some brand new executive makes some silly statement and it's treated as the official opinion of the Xen community.
:-)
;-)), VMI would already be in the kernel.
Disclaimer: Xen hacker, personal opinion
VMware went to OLS and presented a paper demonstrating a VMI interface that runs either Xen or VMware at the same speed as the Xen interface.
I missed OLS unfortunately but I've come to the conclusion that VMI is not the best interface for Xen long term. I should say that I was advocating VMI for a long time previously so this isn't just a knee-jerk reaction.
The problem with VMI is that it hides the hypervisor interface from the guest. Now, if you're VMware, this is ideal. If you're a Free Software project, and your interfaces are evolving overtime, having your interfaces hidden means that there's no pressure to improve them.
The biggest benefit for upstream merge for Xen will be a ton of hackers on LKML auditing the interfaces and saying where they suck and forcing us to change them. You don't want to hide the virtual timer interface behind PIT emulation. You want codesign.
This doesn't mean that Linux shouldn't support VMI. It just means that not all projects should standardize on the VMI ABI.
Xen has never tried to run on a VMware hypervisor, but XenSource went and signed a deal to run on the (future) Windows hypervisor. My opinion is that Xen is a bunch of hypocrites: they complain about how VMware isn't open, then go sign a deal with the least open company of all. Of course, I'm biased.
Please try to separate XenSource from the Xen community. Many of us don't work for XenSource and many of us think that XenSource does stupid things (this being a good example).
Xen wants VMware to adopt the Xen hypervisor interface.
Many of us don't even want Xen to use its own interface. Why would we wish it upon VMware
Imagine if every exec at every company loosely associated with Linux was quoted as gospel for Linux's future.
This is impossible. The Xen interface is too tightly coupled to the Xen hypervisor; it's missing pieces that are necessary to run the VMware hypervisor at reasonable performance. VMware doesn't really care which interface actually proliferates (as in, there will be a layer of interface glue regardless), so long as the interface is good enough. Xen's interface is not good enough. As of two weeks ago, Xen and VMI were the only two interfaces out there.
This is absolutely correct. The Xen interface is not rich enough to support a variety of hypervisors with reasonable performance. Anyone who claims differently is lying to you
Greg K-H's gripe with VMware is that the kernel module isn't open source. Yes and no (I don't want to argue - the code is open but not GPLed), the point is that he's spending more time complaining about Xen and VMware than it would take to actually mediate the problem.
Which is a valid gripe. VMware is going to get the short-end of the stick when interacting with the kernel community because they are doing something that is viewed both as immoral and illegal. There's an easy way to fix that...
Honestly, if there was a single Free hypervisor that worked with VMI (and L4 doesn't count
(Which, thankfully, someone else is doing instead, with paravirt_ops).
Yes, and this is going to be a very long process. I will say too that engineers from Xen and VMware have both been working together surprisingly well on this. There is an active conversation going on in the osdl-virtualization list and on other channels. Despite these stories, Xen and VMware are actually working together.
Finally: I saw more pot-shots about being unable to benc
Subversion only, no real mailing list integration, no real web hosting to speak of. The lack of ads is nice but what I really want is more flexible revision control (I'd like to use mercurial), mail man integration (that doesn't lag like crazy like sf), and a place that I can host a wiki.
:-)
Centralized revision control is so 2002
The FAQ on the XenSource site make it pretty clear that the result of this partnership will be a commercial product. Specifically, one that does hypercall translation from the Xen ABI to the Microsoft Hypervisor ABI.
The Xen ABI is based on a "hypercall page". This is essentially a table of function stubs. The default hypercall page just does hypercalls for each stub. I presume that XenSource is building a special plugin for the Windows hypervisor that offers a Xen hypercall page to the guest. That page would do hypercall translation to the Microsoft Hypervisor ABI. This would require the ability to write a plugin the the Microsoft hypervisor (hence the cooperation) but it pretty much independent of Xen.
Of course, this leads to an interesting question. Since the hypervisor page runs in the same ring (and address space) as the Linux kernel, wouldn't it have to be GPL'd? Certainly, if you agree that binary modules violate the GPL, this can be viewed as nothing but a special purpose kernel module (and therefore a GPL violation).
It's smart that Microsoft is partnering here to limit it's own legal liability.
FWIW, I am not a lawyer and am purely speculating based only on the publicly available information on the XenSource site.
A key point about virtualization (even hardware virtualization) that people miss is that it does not guarentee that programs run as they normally if those programs are timing sensitive. This isn't a new revelation. If you go back to the Popek/Goldberg paper from the 70s, they make it quite clear.
So how do you exploit this to detect that you're in a VM? If you're an operating system, the easiest approach is to disable interrupts periodically and wait out a few time slices. You would then compare wallclock time and see if you're wait took longer than you expected it should. If it did, you're being pre-empted. With interrupts off, that's a sure sign that you're in a VM.
The above is a general solution to the problem. It's funny the author used SVM (a.k.a. Pacifica). SVM has a feature called dynamic attestation. This essentially introduces an unemulatable instruction that one can use precisely for the purpose of determining whether you're in a VM or not.
Python is not something that should be used to write future OS code either.
I think in this case something like Java syntax makes more sense.
I find it interesting that you refer to Java as a high level language. Okay, let's run with it though. You cannot have this problem in Java exactly because functions are not first class types. Java is a very restricted language in this respect. Of course, it's not entirely type safe. Consider:
class A { public int foo() { return 0; } }
class B extends A { }
class C extends A { }
public class test {
public static void main(String args[]) {
A a[] = new B[20];
a[0] = new C();
System.out.println(a[0].foo());
}
}
This gem will compile with no compile time error (even though it clearly breaks the type system). Java arrays are quite silly semantically. A good example of a really strictly typed language is either ML or Haskell.
The problem (if there is one) lies with the language, not the compiler, since both of the above lines are legal C code.
Solutions to this kind of problem probably involve both a movement towards higher level languages (which are typically more verbose and don't allow low-level memory manipulation)
I think we can both agree Python is a higher level language. And guess what:
import os
if os.getuid() != 0 or os.geteuid = 0:
is completely valid. It's not high level vs low level languages here that's at issue. It's static verses dynamic typing and more specifically, strict verses weak static typing. If 0 wasn't treated so specially in C (it's the only numeric literal that's directly comparable to a pointer) this wouldn't be an issue.
Unfortunately, C++ made it even worse since the standard mandates that NULL is defined as:
#define NULL 0
Instead of at least:
#define NULL (void *)0
OSS people need to think more marketing driven.
No, they don't. Market driven is boring. People don't work late at night in their free time to do boring things. Successful Open Source projects are all about having fun. Open Source developers are weird and quirky. The only people who care about Open Source being "market driven" are people who expect Open Source to be exactly like commercial software but free. That's not what it is and that's never what it will be.
It's free, stop complaining.
Something seems fishy about this, though that may be because it's 6:30 in the morning as I begin to write this. /me waves hands
:-)
Well, it's sort of fishy as 100 choose 2 returns pairs that may, for instance, be (a, b) and (b, a). You could remove duplicates by dividing by two but it's not going to change very much in the resulting answer. The key point to take away is that it's a very very large number
At first glance, I'm not sure HOW this is an "unsolvable" problem. Would I not just select and group 100 students at random then rearrange the pairs as I found incompatibilities? Can someone clue me in to what I'm missing here?
... * 1. Since we're trying every possible combination, this gives us:
7 66515247971418153526438677698477539372878051288400 0
What makes a problem NP is not whether it's solvable but rather how long it takes to solve. The algorithm you propose is a search algorithm. Consider what would happen if your list of incompatible students was so large that within the group of 100 students you randomly choose, there is not a single possible arrangement of pairs. This means you would have to choose another group of 100 students. It's a minor refinement but an important one.
Now consider if that list was so large that there was only a single possible group of 100 that contains an arrangement of pairs that worked. Now consider that within that group of 100, there was only one good possible arrangement. If you're very unlucky, and you choose these set of 100 and arrangement of pairs last, you have to try every possible combination before finding the right one. Okay, so what?
Lets see how many possible answers you'd have to try. Within a group of 100 students, there are 100 choose 2 possible arrangements. There are 400 choose 100 possible choices of 100 students. n choose k is really n! / (k! (n-k)!) where n! is n * (n - 1) *
[400! / (100! 300!)] * [100! / (2! 98!)]
Your standard calculator is not going to be able to solve this one but if you have an arbitrary precision calculator (like bc), you get:
1109718121819397093151989141664840784648478532850
Which is an awfully large number. That number is so large, in fact, that even if you have a computer that could check one possible solution with every electron in the universe, until the Sun supernova's, you'd still not find the answer.
Now, that depends on really bad luck. You can construct problems though that given average luck, you would not find the solution in the lifetime of the universe. This is what cryptography is based on.
Compare this to a standard sorting algorithm. To sort the list [3, 4, 5, 6, 7, 8, 9, 2, 1, 0] given a crappy algorithm like bubble sort requires n*n = 100 computations. You can solve this problem the same way using search though. You merely have to randomly arrange the list in every possible way and check to see if your random arrangement is sorted. There are n! possible arrangements of a list of n elements so there are 10! = 3628800 possible answers to search. You can see that even a crappy algorithm like bubble sort is much better than search.
The difference is even greater with larger lists. A problem that is only solvable via search is considered NP. A problem that is solvable with an algorithm in polynomial time (n*n is a polynomial) is considered P. The N in NP stands for non-polynomial.
So the problem here is whether there exists a polynomial solution for these set of problems that we've labelled NP. What makes this even more significant is that it has been proven that if we find a polynomial solution for one NP problem, we can create solutions for any NP problem. A lot is riding on the lack of existence of a polynomail solution for NP problems. If someone where to prove that there are indeed polynomial solutions to NP problems it would be earth-shattering. After the initial shock, it would also open up a whole new world of mathematics since a lot of things we didn't think were possible to do efficiently became possible.
I noticed that whenever virtualization comes up, no one ever mentions CoLinux.
That's because no one really takes it seriously. It's not that it's not novel or not interesting, but the Linux guest runs as a *kernel thread*, not as a process. This means the guest has as much access to hardware as the host. It relies on a very well behaved Linux guest to not bring your system to a screaming halt.
If CoLinux ever adapts to run the Linux guest in a lesser ring (perhaps 1 or 2) then it will be considerably more interesting.
OK guys, now I'm confused. WTF is going on here? Have the Xen people been bought my Microsoft?
d iscloses-virtual-hard-disk.html.
No, TFA is just misleading. A while back, Microsoft documented the VHD format under an anti-GPL license (similar to what they did with CIFS a while ago). XenSource, the commercial company trying to make money from Xen, simply downloaded this license after agreeing to the terms and announced that their product, XenEnterprise, will support importing VHD hard disks.
There is no Microsoft/Open Source cooperation here. See http://www.virtualization.info/2005/10/microsoft-
The preview feature is useless because 1) it's not on every page that comes back and 2) it's not updated live.
1) Just because it doesn't come for every site doesn't make it useless. It clearly has a use for pages that it works on. It may be less useful but it not useless.
2) The fact that it's not updated live does not make it useless either. It is quite useful, for instance, in quickly determining if a site is a link-farm or is going to load some awful flash piece of crap.
I think that's great. Just a while ago Dapper got a new urine-colored Human theme,
I think you've been eating too many carrots if that is the color of your urine...
Esclation of Privileges is a vulnerability, the last time I checked.
According to whom? If it isn't, why do so many programs go to such great trouble to reduce privilege level when running? Hell, why do we even have users in the first place?
But they are in ogg format. Didn't even know there was a ogg video format. As long as the Linux world continues to alienate those using Windows, I don't care about it.
Then disable the Linux section in your slashdot main page and stop complaining.
Why couldn't they just standardize on Xgl? It works *today*. Aigl doesn't even support my nvidia card right now.
:1. Plus, some things are noticably choppier (web browswer scrolling).
It doesn't Just Work. The integration with Gnome is very poor (you can do awful things like hide window bars under the panel--doh!). It uses a whole new Window Manager (WM's are tricky to get right so quite a few apps don't work now). emacs doesn't start for me under Ubuntu (the xrdb seems to be screwed up).
Plus, Xgl for ATI cards is very buggy (due to the ATI drivers) and require running Xgl on
Some of the effects are cool in Xgl (the window scaling for instance) but some are annoying (like the new switcher). Being able to shut of GL effects *without* restarting X is a big deal. If I'm just trying to program, I don't want wobbly windows or any of that stuff.