Slashdot Mirror
First Look At VMware's vSphere "Cloud OS"
snydeq writes "InfoWorld's Paul Venezia takes VMware's purported 'cloud OS,' vSphere 4, for a test drive. The bottom line: 'VMware vSphere 4.0 touches on almost every aspect of managing a virtual infrastructure, from ESX host provisioning to virtual network management to backup and recovery of virtual machines. Time will tell whether these features are as solid as they need to be in this release, but their presence is a substantial step forward for virtual environments.' Among the features Venezia finds particularly worthwhile is vSphere's Fault Tolerance: 'In a nutshell, this allows you to run the same VM in tandem across two hardware nodes, but with only one instance actually visible to the network. You can think of it as OS-agnostic clustering. Should a hardware failure take out the primary instance, the secondary instance will assume normal operations instantly, without requiring a VMotion.'"
With no delay at all? Somehow I don't believe it - there is always delay, but I wonder if it is "significant" enough to be noticed by an end-user.
FT only supports a single vCPU from my understanding... Not too many people running single CPU VM's, at least in my experience...
You should be running single vCPU machines by default and only scale CPU's if absolutely necessary and if the app is fully SMP aware and functional.
http://virtualize.wordpress.com/
If your apps are not SMP aware, WTF at you using?
Multiple CPUs has been the standard for servers for at least a decade in x86 gear.
Check out the Kemari and Remus projects, which allow precisely the same in Xen environments. In essence, it's a continual live migration (vmware people, think continual vmotion) that resumes virtual machine execution on the backup node if the origin node dies. Very cool tech. The demonstration involved pulling the plug on one of the nodes. For more information just search, there are code and papers and presentation slides galore.
He didn't say to use a single vCPU for a non-SMP aware app, he said to use a single vCPU for all application loads. For SMP aware apps, adding another virtual CPU is a scaling option. If you have non-SMP aware apps, then you need to find another solution, like migrate to a host with faster cores.
It makes sense. If you have 32 workloads and 16 cores, don't add the overhead of making 64 virtual vCPUs, 32 will use the host resources more efficiently as long as one app doesn't need the power of more than one core. If it does, give it to the guest only when it needs it.
Multiple CPUs has been the standard for servers for at least a decade in x86 gear.
Seriously, even Windows NT 4 had SMP support in 1997.
I don't know what year Linux first had support for SMP, but the 2.0 kernel supports SMP, apparently even on a 486. Just imagine a Beowulf made of 486 class SMP machines!
/^([Ss]ame [Bb]at (time, |channel.)){2}$/
If your apps are not SMP aware, WTF at you using? Multiple CPUs has been the standard for servers for at least a decade in x86 gear.
Yes, but it doesn't work in VMs the same way, at least not in VMWare. On a loaded system you often find a single-vCPU VM will out perform one with more than one vCPU, in fact if you can spread your app over multiple machines you are generally better off running two single CPU VMs instead of one dual-CPU one. This is true no matter how many physical CPUs/cores you have available.
Why is this? Because a single CPU VM can be scheduled when-ever there are time-slices available on any physical CPU/core (though a good hypervisor will try not bounce VMs between cores too often, as this reduces the potential gains from using the core's L1 cache and (on architectures where L2 cache isn't shared) L2 cache too), but a dual vCPU VM will have to wait until the hypervisor can give it timeslices on two CPUs at the same time. If this is the only VM that is actively doing anything on the physical machine (and the host OS is otherwise quiet too) this makes little difference aside from a small overhead on top of the normal hits for not running on bare metal, but as soon as that VM is competing with other processes for CPU resource it can have a massive negative effect on scheduling latency.
I just am shocked to hear of any real server apps that are not smp aware.
If you have any boxes with one vcpu then it can't take another cpu when it needs it. Great, now you have 16 boxes and all have one cpu, too bad that none of them can soak all 16 when the others are totally idle. What a wonderful way to waste hardware.
This is why guest priorities are useful. Give them all a number of cpus that they actually needand let the priority sort it out. You will find that is what the individual OSes are doing anyway via their schedulers. There are not going to be any hard and fast rules about it, other than if you have 1 vcpu you should have more on that box since the guest overhead is going to kill you on hardware consumption if that is your practice.
I had an idea at some point of a distributed app, similar to SETI@Home, that people would run on their computer. These computers would form a cloud which would support creating VMs that could run arbitrary code. If one app is currently running your code, and the computer it's on goes down, your code would continue to run on another one. If everyone runs it, it would be a huge pool of computational power. Then you could run crazy things on it. Then, profit! Anyway, is this a step in that direction?
Seriously, even Windows NT 4 had SMP support in 1997.
I don't know what year Linux first had support for SMP, but the 2.0 kernel supports SMP, apparently even on a 486. Just imagine a Beowulf made of 486 class SMP machines!
Making good use of multiple CPUs requires more than just OS support.
, but a dual vCPU VM will have to wait until the hypervisor can give it timeslices on two CPUs at the same time.
Then their hypervisor is broken.
It should be possible for A dual vCPU machine to have vCPU1 and vCPU2 be two timeslices on the same real cpu if need be.
How many hardware failures are actually characterized by a complete 100% loss of communication (as you'd get by pulling the plug)?
Don't CPU and Memory failures tend to make the computer somewhat unstable before completely bringing it down? How would vSphere handle (or even notice) that?
Even hard disk failures can take a small amount of time before the OS notices anything is awry (although you're an idiot if you care enough about redundancy to worry about this sort of thing, but don't have your disks in a RAID array)
-- If you try to fail and succeed, which have you done? - Uli's moose
Basically all server apps are SMP aware. Do you think IIS or Apache only have one worker process?
Perhaps that MSSQL and Postgresql only use one thread?
One of the statistics measured by virtualcenter is the lag you're asking about.
The first hit on google images should give you a good idea.
In practice, I don't know... I imagine that the secondary instance will still receive network traffic bound for the cluster, so it'd probably be perceived as a hiccup when the primary one goes down, which is good enough for most services.
Boot Windows, Linux, and ESX over the network for free.
Like everyone else pointed out, it's a VM in lockstep with a 'shadow' VM. This is not just 'continuous VMotion'.
If something happens to the VM, the shadow VM goes live instantly (you don't notice a thing if you're doing something on the VM).
Right after that, the system starts bringing up another shadow VM on another host to regain full FT protection.
This can be network intensive, depending on the VM load, and currently only works with 1 vCPU per VM. Think 1-2 FT VMs per ESX host + shadow VMs.
You'll need recent CPUs that support FT and have an VMware HA / DRS Cluster set up.
So if you've got it, use it wisely. It's very cool.
-- Robi
, but a dual vCPU VM will have to wait until the hypervisor can give it timeslices on two CPUs at the same time.
Then their hypervisor is broken. It should be possible for A dual vCPU machine to have vCPU1 and vCPU2 be two timeslices on the same real cpu if need be.
Which would kill any benefit of running SMP in the VM anyway, if it were possible.
My understanding, which may be out of date, is that this is not considered a good idea as timing issues between threads on the two vCPUs if scheduled one after the other on the same core could potentially cause race conditions. And if not that serious, the threads on the vCPU that gets the first slice of the real core could be paused waiting for locks to be released by threads the guest OS has lined up to runon the other vCPU. This is an explanation that I have seen given as to why VMWare would not allow you to do virtual SMP on a single-core-single-CPU host machine (i.e. emulating SMP in the guest by giving two or more vCPUs alternating time on the only physical core).
The article mentions an inability (for the "pre-released" version) to PXE boot. If he's talking about booting for installation, then he's 100% wrong. The ESX beta/RC (build 140815) will, indeed, boot and install over a network. It's different from 3.5 so you'll have to adjust your commandline and/or kickstart. They use "weasel" instead of "anaconda" and that runs inside the service console. Short answer... "method=" becomes "url=" -- with a properly formated URL, eg. url=nfs://server/path/to/esx/. It's a much larger boot enviroment -- 80MB -- so it takes longer to boot, and from my half dozen installations (I'm only testing on 2 machines), it takes substantially longer to install 4.0 than 3.5. (my 3.5 installs take 2.5mins.)
Try supporting synchronization of two virtual machines over a network/SAN when you also have to deal with SMP. Gets hard.
“Common sense is not so common.” — Voltaire
Yea, but you also get the overhead of two schedulers (the host and the guest) and two systems moving thread context from core to core, which is an expensive operation. Most VMware systems are pretty heavily oversubsubscribed in terms of cores. Its not uncommon to have 60 guests on a 16 core host. If all 60 guests have 4 virtual CPUs, you do get the advantage that one guest can expand out to consume about a quarter of the total host CPU power, but you also have the cost of guest SMP switching even though you are using an average of 0.25 cores per guest.
It wasn't them that I was thinking of specifically.
There are plenty of applications of databases (particularly MIS-type things) which don't really lend themselves to multithreading that well in the first place.
(BTW, Postgres only uses one thread per query and changing that, the last time I checked, was part of some significant work which is only just taking place right now)
SQL server is a great example. Most of the time on dedicated hardware you might have one SQL server with several Databases and maybe even SQL instances. These will happly share the multiple CPUs torun seperate apps concurrently, but most queries in SQL execute in a single thread.
With VMs, as you dont have to pay $$$ for more hardware, only software licences, we have seen more customers simply provision another single CPU SQL server in VM for general duty work. These would generally have no tuning in the applications or queries for multithreading, so they work fine in single CPU. Disk IO tends to be the biggest bottleneck.
So what they're saying is that they don't believe in operating systems, but acknowledge that they might exist?
Think of how stupid the average person is, and realize half of them are stupider than that.
And yet most processing workloads in apps I work with are single-threaded and not terribly scalable. The reality is that >1 CPU is exponentially more difficult to code for (at least so the excuses I hear from developers goes).
However, if your application can sit behind a load balancer and runs a single thread, why NOT have n*single CPU servers? Of course, that sort of takes away from the instant failover advantage. Ho hum :P
You obviously don't deal with many vertical apps. Most of them are not SMP aware, or tend to sit there hogging a single CPU/Core pretty much all the time. At least, this is true in the Windows world... UNIX is quite different, but there's a dearth of the applications my users want to run.
No, just 'cos I do it for a living doesn't mean I have to like it :)
I'm on the FT team at VMware and just wanted to provide some additional information on FT requirements. You can also find out more about FT at: http://www.vmware.com/products/fault-tolerance/
VMware collaborated with AMD and Intel in providing an efficient VMware Fault Tolerance (FT) capability on modern x86 processors. The collaboration required changes in both the performance counter architecture and virtualization hardware assists from processor vendors. These changes could only be included in recent processors from both vendors: 3rd-Generation AMD Opteron(tm) based on the AMD Barcelona, Budapest and Shanghai processor families; and Intel® Xeon® processors based on the Penryn and Nehalem microarchitectures and their successors.
The current set of VMware FT supported processors are:
Intel® Xeon® 3100 Series, Wolfdale (UP)
Intel® Xeon® 3300 Series, Yorkfield
Intel® Xeon® 5200 Series, Wolfdale (DP)
Intel® Xeon® 5400 Series, Harpertown
Intel® Xeon® 7400 Series, Dunnington
Intel® Xeon® 5500 Series, Nehalem
AMD Opteron(tm) 1300 Series, Budapest
AMD Opteron(tm) 2300 Series, Barcelona (65nm, DP) and Shanghai (45nm, DP)
AMD Opteron(tm) 8300 Series, Barcelona (65nm, MP) and Shanghai (45nm, MP)
VMware maintains a KnowledgeBase (KB) article that provides a current list of supported processors, see http://kb.vmware.com/kb/1008027.
You can download a utility, VMware SiteSurvey at http://www.vmware.com/download/shared_utilities.html to check if your configuration can run VMware FT.
While we understand the end user's desire to be able to use VMware FT on as many processors as possible, VMware's goal is to guarantee that Fault Tolerance works reliably.