Slashdot Mirror
The Trouble with Virtualization - Cranky IT Staffs
lgmac writes "A new survey on the results of Enterprise use of virtualization shows that the process is seeing wide and appreciative use. Technical hurdles are obviously the biggest problem facing corporate IT shops. Just the same, political squabbles among IT staffers fighting for turf after being forced to work together in new ways seems to be a going concern as well. 'Technical woes rank higher--to be expected when CIOs deploy a new technology such as virtualization. However, the politics pain many of you. Remember, virtualization not only asks people to cede some control over their physical server kingdoms, but also asks IT experts from different realms to work more closely together.'"
34% of surveyed companies have been running virtualized desktops? Putting aside that that number doesn't seem to square with the "Virtual Desktops a Hard Sell" table below, does that seem likely?!?
What I'm listening to now on Pandora...
Depends on the virtualization solution. Vmware has a product called Vmware consolidated backup. You can also load agents on the vms just like a physical server and back them up. You could also use things like mirroring and snapshotting to back them up at the storage layer. We used a combination of all 3 at my previous employ. Really depended on the virtualized box, what needed backing up, how often, etc.
On the VMware side, there's several options. VMware's Consolidated Backup does exactly this. Also you can look at ESX Ranger.
It may be more practical to back up the system from within the VM, i.e. treat it as if it weren't a VM. By definition this will be on a live system.
Another option:
Have your VM use a checkpoint disk. Once a day shut down the VM, merge the changes from this week into the checkpoint disk, and restart the VM. This may take anywhere from a few minutes to tens of minutes. Restart the VM. Back up the checkpoint-disk image.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
How do you ensure that the VM supervisor fairly and efficiently allocates resources to the VMs? The mainframe people put a great deal of work into this area. One badly behaved VM shouldn't be able to degrade the performance of the other VMs.
Mea navis aericumbens anguillis abundat
*facepalm* I sometimes forget how stupid people can be.
Personally, what I've found to work great with virtualization is consolidating all the dozens of little low-load servers. It helps with power consumption and heat output, as well as hardware costs. For a major company-wide high-load system, virtualization is absolutely not what I would be looking at. It's also fantastic for testing environments.
I don't know about you, but my servers run on the power of cotton candy and happy thoughts. -Anonymous Coward
With HA and Clustering capabilities offered by many virtualization solutions you could end up taking a physical server and resources that weren't redundant and through consolidation efforts end up with more redundancy than before. It's all in how the solution is designed and knowing when to use virtualization and when not to use virtualization.
There are pros and cons to virtualization.
You can build a beefy SAN and have the vm's directly access luns on the SAN, so no virtualization of disk I/O. If the vm gets corrupted or the physical server it is running on fails, it can very quickly start running on a seperate server or blade. With a traditional approach, you might be reaching for backup tapes, swapping drives over into a spare chassis, or worst case reloading an O/S while everyone wonders what is going on. Virtualization gives you some very interesting options for disaster recovery.
VM's can share excess resources that normally are totally wasted on traditional server farms. I have worked at several fortune 100/500 companies and I have done capacity planning for cpu and disk resources. Servers are usually very bursty with periods of high demand, followed by long periods of idleness. Obviously this depends on production demand, type of server and when jobs are run, etc etc. Regardless, a lot of CPU power is just spending time in NOOP's. With a virtual environment you can run a lot more vm's on the same physical hardware but the hard part is to predict demand and make sure that the vm's will have the cpu resources when they need them. With a little tuning and monitoring, you can usually find this balance.
I can sit here and write for hours the pros and cons, but I think anyone that has truly used this in production use is aware of these. Most of the people I see that are "against it" are people afraid of change and don't truly understand the technology. BTW, I am not saying that ALL servers should be virtualized! Nobody every said you have to run everything on a virtual server cluster. You can run a mixed environment with your heaviest of applications running on dedicated hardware.
The biggest problem I have seen with virtualization is trying to get enough RAM into the server to adequately utilize the cpu resources available and to get software vendors to "support" their products when running in this environment (many have disclaimers to say they wont support virtualization, but this will change in time or they will go out of business). Another problem is that it is really expen$$ive if you want to go with the best of the breed solution (ESX Server) just for the virtualization software itself. Two Dell 1855R chassis with 10 blades each, a capable SAN and VMware ESX/VMotion is a beautiful thing!
Another approach is to make your physical servers do MORE tasks than they did before to ensure they are utilizing CPU resources. So do more with less physical servers. There are a lot of cons with this approach and most traditional IT staff would prefer to just rack a new server for each new project rather than trying to shoe-horn applications onto existing production servers and potentially disrupt them.
Hold up, on a cisco router or switch "vrf" is used for mpls process separation. It doesn't virtualize multiple instances of IOS.
Say you have a need to add another fax board(or whatever) to the virtualized x86 server, to find that they stuck some mission critical Virtual Environment on the Server and It CAN'T come down for another 2 weeks.
:) If you need to do work on a physical server [say hardware replacement, firmware upgrade, etc], you put that server into maintenance mode. VMWare will automatically and **transparently** migrate the running VMs onto the remaining servers. You can then power off the physical server, do whatever you need to do, and power it back on. When it comes back up, take it out of maintenance mode, and VMWare will automatically start migrating VMs back onto it to balance the load.
Aside from another poster's excellent point about not virtualizing servers that require specialized hardware, you're missing another point of the virtualized servers.
In the case of VMWare ESX server, you'd use VMotion to solve this problem. Say you have a cluster of 3 or 4 physical servers running some number of VMs. Hopefully you're not dumb enough to have all those servers running anywhere near 100%.
Nobody will notice the fact that their mission critical server just moved from one box to another. Worst case, if your servers were already a bit too heavily loaded, some applications will slow down a bit while you're doing maintenance, but a temporary slow down is a lot better than having an application completely down every time you have to upgrade firmware or replace a stick of RAM.
As an added bonus, if you so choose, VMotion can automatically balance the VMs at all times, so if one particular VM is suddenly requiring a large amount of resources, VMotion can migrate it to a less heavily loaded box, or migrate other less needy VMs to another box to free resources for the VM that needs it. This is great for handling short-term usage fluctuations, or can even be scripted to adjust for known, regular usage peaks.
That's not to say there aren't downsides to virtualization, but the situation you described isn't one of them.
Nothing to see here
ESX3 is Linux based however ESX3i is not. It is a pure hypervisor checking in at about 32MB.
I think your math is flawed. Your 500 physical servers just went to 500 virtual servers. Each one has a dependency that is now harder to dissect and is more abstract from the hardware.
Sure you may have a smaller room - but you definitely have more complexity. Now you have servers moving all around, because you really can't tell what their true capacity is on the physical server - it just looks like CPU or memory is getting busy. Now you need to move it. In pops VMotion. This is very fancy ooo-aaah stuff. You start doing this once, twice, eighty times a day (obviously depending on the size of your environment) However, soon you start with what I call "VMotion Sickness".
See, Vmotions create two problems. 90% of the time they are just moving a problem resource to somewhere else, creating a never-ending game of whack-a-mole, AND at the same time it consumes a serious amount of resources across your entire virtualization environment.
What you've done is add COMPLEXITY to those 500 servers by pushing them onto 100 physical boxes. You've also magnified risk of physical hardware problems so maintenance must be much more rigorous. And you've now added a huge learning curve to your entire team to learn how to triage any problems and avoid whack-a-mole.
Full disclosure: I work for a systems management software company, http://www.hyperic.com/, that specializes in managing virtualized environments and talk to these shops every day. Also, while I am at it - one of our customers, http://www.mosso.com/, a clustered hosting provider that is a division of Rackspace built a great case study on managing a 100% virtualized environment. And for the record, they were able to keep their staff the same, which they thought was a big achievement. They had actually thought that virtualization would add so much complexity, they would have to ADD staff to maintain SLAs. The case study can be found here: http://download.hyperic.com/pdf/Hyperic-CS-Mosso.pdf
Friends dont let friends virtualize production systems with any 'native' operating system.
Get a hypervisor that doesn't suck so bad it needs another OS to run, and your windows and linux guys will get along much better.