Slashdot Mirror
Red Hat & AMD Demo Live VM Migration Across CPU Vendors
An anonymous reader notes an Inquirer story reporting on something of a breakthrough in virtual machine management — a demonstration (not yet a product) of migrating a running virtual machine across CPUs from different vendors (video here). "Red Hat and AMD have just done the so called impossible, and demonstrated VM live migration across CPU architectures. Not only that, they have demonstrated it across CPU vendors, potentially commoditizing server processors. This is quite a feat. Only a few months ago during VMworld, Intel and VMware claimed that this was impossible. Judging by an initial response, VMware is quite irked by this KVM accomplishment and they are pointing to stability concerns. This sound like scaremongering to me ... All the interesting controversy aside, cross-vendor migration is [obviously] a good thing for customers because it avoids platform lock-in."
I love to see things like this that give me a greater freedom to migrate off the major players.
The real beauty of this will come when the system automatically moves VMs to machines in case of hardware problems or when a system is underutilized. It would let you power down servers during non-peak times and save oodles of cash.
---- aut viam inveniam aut faciam
The fact to highlight is that the migration was done of a live VM without disrupting the VM's operations.
for the last time people, I am "frodo from middle eaRTH", not "middle eaST".
Xen supports this feature since Xen 3.3, it is called CPUID: http://www.nabble.com/Xen-3.3-News:-3.3.0-release-available!-td19106008.html No real breakthrough here...
Real magic would have been demonstrating a move between ANY processor architecture - Power, SPARC, x86_64 etc..
Between x86 processors is nice, but not unexpected.
but is it secure yet?
Open source is for morons.
Only Apple has the engineering know-how and skills to pull of something like this. The fact that they have not done so to date is a clear indication that it is impossible.
Go to 4:05 in the video. On the far left, you can see from the blue intel line that the guest is running there, then they migrate, and the blue line goes to the idle point, and the orange line starts taking the load. But NOTICE, the AMD line is consistantly higher than the intel line was. I'm no intel fanboy... or AMD. I have both intel and amd servers in my racks. I just thought it was interesting, and I'm surprised they let the video go out like that.
Do not meddle in the affairs of sysadmins, for they are subtle, and quick to anger.
can this, theoretically, be done with a mixture of big-endian and little-endian architectures?
This is completely trivial. You simply have to mark the VM with the architecture of its code. Then each host contains both a virtualization layer (à la vmware) and a multi-platform emulator (à la qemu). If the VM matches the architecture the host is running on, you use the virtualization layer, if it doesn't, you use the emulator.
As for moving between AMD64 and Intel 64 (for example), the VM has to emulate the few instructions that differ and virtualize the rest.
Of course, cross-architecture migration is not that useful since you have an emulation penalty. It is much simpler (and cheaper) to do everything on x64.
Declaration: VMware support engineering here, but speaking strictly on my own behalf.
The stability issues are justified if you consider all types of VMs. Windows 2003, default RHEL5 kernels etc use more than the basic set of assembler instructions (disk IO code uses MMX, SSE etc).
We can compile a kernel for strictly 486 CPUs and demonstrate migrations between AMD and Intel using extensive CPU masking: http://kb.vmware.com/kb/1993
We've also known that mismatched CPU stepping makes the VMs unstable. This is because instructions suddenly run faster or slower compared to the front side bus, not all of Linux and Microsoft code has been tested against that. You can happily try it and a lot of our customers succesfully do. Some get BSODs and kernel oops. This is not our fault.
If you virtualize the instructions more (bochs?) you can of course move the VM anywhere including a Linksys router's MIPS chip. At the cost of speed of course.
Lastly, why would we want to keep customers stuck to one CPU vendor? We've software vendors.
"Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
Not to diss the acheivement which is cool. It does require newer processors with the special VM extensions. So it may commoditze future CPUS. Also KVM requires QEMU and many running VMWARE depend on a tested solution that is delivered complete from the vendor. And VMWARE is propably looking at the source and will have it or something similar in future builds.
The VM software vendor becomes "the major player".
As The Who's so insightfully titled song said "Meet the new boss. Same as the old boss."
Deleted
The point of virtualization is to isolate the hardware from the software - I fail to see how this is unique other than it being done "live" (which just means the VM is suspended, and the state of everything moved to the new machine and the VM resumed). Nor how it cna be impossible - while the x86 has many extensions, it's still a well-specified architecture with specific behaviors.
The real trick is if an application is using features not present on the other architecture - e.g., an AMD virtual machine migrating to an Intel one while running applications use 3DNow instructions (which don't exist on Intel CPUs). Or perhaps an old 16-bit application running on a 32-bit VM under a 32-bit OS migrating to a 64-bit VM (since you can't do real mode or other legacy things in x64 mode) and continuing without a hitch... (Maybe it's a VM running MS-DOS, say?)
After all, all x86 are the same. MMX extensions get emulated on AMD, Linux distro's run on both processors without recompiling, the kernel handles calls and most likely an Apache server is not going to call the special media extensions. It would be interesting to see this happen in an environment that has been optimized and is using certain incompatible extensions (like 3DNow!) eg. a computing cluster.
If you abstract enough and emulate a processor you should even be able to move between architectures but the overhead of emulation wouldn't make it very cost effective.
Custom electronics and digital signage for your business: www.evcircuits.com
FWIW, KVM live migration has been capable of this for a long time now.
KVM actually supported live migration of Windows guest long before Xen did. If you haven't given KVM a try, you should!
This is a demo of a Live migration, no shutdown or reboot involved. Xen does not support the live migration of a running VM between an AMD and Intel server. Watch the video, they are running a video in the VM that keeps playing during the migration. Very impressive stuff.
Kindness is the language which the deaf can hear and the blind can see. - Mark Twain
It's worth noting that VMware have been a huge contribution to the Linux-society, giving corps a very good reason (â$£) to migrate, thus including important pawns in the future of Linux. I for one believe that VMware was wrong, but that it's an honest mistake. There's no use in poking on VMware for this one, hopefully they'll help lift the technology even higher along with their competitors.
You've lost this round VMware, but the match isn't over yet!
I am the lawn!
I don't know if this will help AMD sell more procs. I like AMD, but Intel's stuff is by far faster these days. Still, Intel's procs are nightmarishly expensive compared to AMD, and the difference in price/performance seems disproportionate to me.
Shows that AMD is the better company. Intel just buys and kills everything in it's way with it's evil black market and under the counter deals...
Let me clarify before people jump down my throat... OpenVZ (www.openvz.org) is OS Virtualization (aka containers) and NOT machine / hardware virtualization... so it can only run Linux on Linux... but it has been able to do live migrations from one processor family to another since they initially added checkpointing. OpenVZ is fairly CPU agnostic and it has been ported to a number of CPU families. In fact the project leader recently ported it to ARM (Gumstix Overo). See: http://community.livejournal.com/openvz/24651.html
Scott Dowdle
www.MontanaLinux.Org
This migration won't work for systems that employ advanced JIT code generation, such as Java. Modern production JVMs, like Sun's and IBM's, will create native code on the fly - and they will produce code that's ultra tuned for the specific processor that is running. This means using the best instructions available (like SSEx), and also fine-tune various behaviors, e.g. GC can be tuned for the L1/L2 cache sizes, and locking can be tuned to factors like number of CPUs/cores/hardware threads - so for example, if it's running on a uniprocessor/single-core machine, the JVM will simply not emit memory barrier instructions for memory model consistency.
And it's not only Java, we have an increasing large number of JIT compilers that may employ similar tricks: Microsoft .NET (CLR); Flash 9+ (Tamarin) for ActionScript; Mozilla TraceMonkey and Google V8 for JavaScript; new LLVM-based runtimes for other languages... the list is only growing. Even for traditional static-compiled languages, some apps can have multiple shared libs compiler for different CPU levels, and choose the best lib at startup.
The only way I see around this problem is making ALL these runtimes and applications migration-aware. Each process should be notified before the migration, initiate some pre-migration task, and after the migration, being notified again to resume work and if necessary perform some post-migration step. Specifically for Java, the pre-migration would need to "park" all threads in OSR safepoints, then free all JIT-generated code; and in the after-migration, retune/config itself for the new CPU, then unpark the threads - that would resume execution in interpreted mode until the JIT compiler recreates all native code for the new CPU. Fortunately this is relatively simple to do in JVMs, because all necessary plumbing is preexisting (safepoints, on-stack replacement... required for advanced GC and dynamic optimizations). And once a new JVMs are enhanced with this feature, thousands of Java apps become magically migration-aware. Could be harder though for other runtimes.
Still, very hot technology, just not as easy as we can imagine to get right and compatible with all applications.
AMD is the first to technological breakthrough and all Intel can do is copy the technology and overclock it to do better on benchmarks.
AMD - First to create lower clock speeds with same or better performance to Intel's higher speeds.
AMD - First (and only) to TRUE dual and quad core technology (Intel does not use logical cores).
AMD - First to 64-bit.
Of course other smaller chip makers have done these sorts of things first, but they don't compare to the Intel/AMD dominance and consumer marketplace.
I've seen this done before by masking the cpu flags so as the VM only sees the lowest common denomination of features of a group of CPUs across which it can migrate.
VMWare have been unable to make this feature stable in their product while others like the commercial products based on KVM and also Citrix Xen have managed to get this working to a level where they are confident enough to do live demos (rather than slideware). There was a video of this linked over on the 360is blog last week.
AG
Look for Intel to provide "Intel Genuine Advantage" that makes it impossible to migrate a VM, under any circumstances, with any degree of success.
I'm not saying who, but this is a big old badazz Bitch slap to a large proprietary software vendor famous for locking customers in, and feeding them anything it wishes, including animated paperclips and other stupid junk. Suddenly their protected world with its high walls are looking like an open beach with the tide rolling over walls made of sand.
With all the talk about virtualization in the last couple of years, I'm a bit surprised that I haven't seen major talks about live migration capabilities at application level.
I'm not talking about cluster capable apps, but being able to run an app on one server, and then migrate it.
Even the capability of FreeBSD jails, Solaris containers, OpenVZ, etc. to migrate live would come closer to live apps migration.
There's always a good reason to virtualize at OS level, but ultimately it only comes down to being able to run the application that you need.
home
My company have been successfully migrating VMs from 32bit Intel to 64bit AMD to 64bit Intel for years. We use Linux VServers and OpenVZ. This shared kernel approach to virtualisation is much lower overhead than VMWare, Xen or KVM. We can even run different distro's inside the VMs, the only limitiation is that all the VMs see the same Linux kernel version. So whilst we haven't done this with a hypervisor style VM, for what we want (migrating server images between physical hosts, backing up server images) Linux VServers/OpenVZ is a much better choice anyway.
nt
Redundant Array of I...Intercommunicating D....Devices of Virtual Machines.
It is obvious that the next step is to set up several servers with Virtual Machines on them. Run the same VM in parallel on one or more of them. And if one of the servers goes down - the end user will not notice this, because his virtual machine was be mirrored on all those other servers. Just like hotswap in RAID HDDs we will have this capability with Virtual Machines. It's just a matter of time.
And if someone is stupid enough to try to patent this - he can't because it's blatantly obvious, and I doubt that I am the first person the present this idea.
#
#\ @ ? Colonize Mars
#