Virtualizing Cuts Web App Performance 43%

czei writes "This just-released research report, Load Testing a Virtual Web Application, looks at the effects of virtualization on a typical ASP Web application, using VMWare on Linux to host a Windows OS and IIS web server. While virtualizing the server made it easier to manage, the number of users the virtualized Web app could handle dropped by 43%. The article also shows interesting graphs of how hyper-threading affected the performance of IIS." The report urges readers to take this research as a data point. No optimization was done on host or guest OS parameters.

Comments

Bogus Test

[Anonymous Coward]

Who uses VMWare Server in a production environment anyway? We run all of our Web services, Exchange servers and SQL databases in VMWare's Virtual Infrastructure 3. VMWare Player and Server are only ment for lab evironments and low load applications. VMWare even says as much on their website. Either this is just FUD or the author is an idiot. In other news water is wet.

Re:Bogus Test

[Sobrique]

Actually, the company I worked for 6 months back, one of the projects I was involved in was 'VMWare'. Production stuff running on on the ESX servers (which became 'virtual infrastructure') in our datacentre, as a cost effective scalable environment. Yes, we weren't getting 'uber performance' but then again, we were running 150 or so VMs on an 6 server VMWare farm.

One of the other things we prototyped and deployed was 'site services packages' - get GSX (now VMWare Server), stick it on a pair of 2U servers, and attach a storage array to both of them. Then create your 'template' fileserver, DHCP server, print server, proxy, that kind of thing and deploy them to this package. It worked very well indeed - you get a whole new order of magnitude on stability (although to be fair that's in part because we through away the crappy workstations that were doing the 'low intensity' stuff) and was extremely managable, and trivially replacable in the event of a hardware failure.

Performance? No, VMWare isn't that great on performance - whilst it's not bad, in an ideal situation, fundamentally what you are doing is introducing an overhead on your system. And probably contention too. But it's really good at efficient resource utilisation, easy manageability and maintainability.

As an experienced sysadmin, my reaction is screw performance. Let's start with reliable and scalable, and then performance just naturally follows, as does a really high grade service.

Proactive laziness is a fundamental of systems admin. Your job, is essentially to put yourself out of a job - or more specificially, free up your time to play with toys. The best way to do this is build something stable, well documented and easily maintainable. Then your day consists of interesting stuff, punctuated by the odd RTFM when something doesn't work quite right.

Re:Bogus Test

[sammy baby]

Yes: it performs much, much better.

VI3 is actually a suite of products. At the heart is VMware ESX Server, which is actually an operating system in its own right: it runs "on the metal," without having Windows or Linux installed already on the system. It also has a service console operating system which looks suspiciously like a *NIX style operating system, so you can SSH directly to the system, cd into your /vmfs directory and, say, scp disk files over the network. If you wanted to.

However, as a pretty damn safe rule of thumb, no system is going to run faster on equivalent hardware after being virtualized. In a prior job where I was often asked to provide development/test systems, I got phone calls from a lot of people who were bitten hard by the virtualization bug. Whenever someone brought up any issue having to do with infrastructure, no matter how odd or off the wall, they wanted to push virtualization as a solution. I had to explain to them that if your problem is that a web server is slow, the answer isn't to install VMWare server on it, set up two host operating systems, and say, "There! Now I have two web servers." You'd be surprised how pervasive that sort of thinking is, even among people who should patently know better.

Another useful guideline: various types of services are impacted differently by being virtualized. Generally, the best candidates for virtualization are ones that spend a lot of time idle. This is actually more common than you might think - people need a server set up for something, can't put it on a pre-existing system for security/compatibility reasons, so they go out and buy a new system which is ten times more powerful than they need. You can put a lot of these kinds of systems on a single, reasonably powerful ESX server. On the other hand, systems that heavily tax available hardware, especially I/O, are usually much harder to deal with.

Re:Bogus Test

[Bohiti]

Actually, as it's been explained to me, the ESX hypervisor itself is pure proprietary code (and small, too). The Service Console is very readily admitted to be a tweaked out RHEL (3, I believe..). Linux is used to boot, and then (magically?) transfers control of the bare metal to the hypervisor. Linux then jumps into a virtual machine, although it's not presented like a virtual machine, which creates all this confusion.

In the end, the tweaked RHEL that you interact with (ssh, scp) is not the hypervisor, but a VM with special tools that can manipulate the hypervisor.

This is VMware Server and not ESX Server

[Fuyu]

They performed the test on VMware Server not VMware ESX Server which is what most enterprises will use. VMware ESX Server runs on "bare metal", so it does not have the overhead of the host operating system.

Bad data, bad setup

[duncanFrance]

There's quite a lot wrong with their setup.

1) As others have pointed out, they should be running on ESX to get best performance.
2) Physical machine was a dual-proc. How many processors did they assign to the VM?
3) Physical machine had 2GB memory. They assigned 2GB to the VM!! Vmware will take 256MB of this
for itself, so that 2GB visible to Windows will be being swapped.
4) How many disks did the physical machine have, and what was on them?
If e.g. the physical machine had two disks, the VM should have been given two disk files, with each file being placed on a different physical spindle.

You get the picture.

holy cow am I a nerd

[thegnu]

I do like the idea of a variably sized beowulf cluster running a floating number of package (LAMP) servers. Get more clients? Add more VLAMPs. Things slowing down? Add more hardware.

I started getting aroused as I read your post. This is highly disturbing.

Re:Well,

[Mr. Underbridge]

Seriously. I don't know who gave anyone the impression that virtualization was a performance booster. Management improver? Sure. Stability insurance? Why not? But if you don't get that virtualizing your servers imposes a bit of overhead, then you're probably not paying attention.

Well, I think the point was that he attached an actual number to the amount of the performance hit, which is relevant. That's called research; quantifying and proving that which seems 'obvious'.

Re:Well,

[hey!]

Well, it's not a surprise, but it's probably worth quantifying.

Here's a question: what is more available: hardware or skilled system administrators? Obviously hardware.

Here's a common scenario: you've set up a system to provide some useful package of services. How do you let other people duplicate your success? (1) tell them what hardware they need and (2) have them install and configurethe software on their hardware. Guess which item involves the most cost in the long run?

The hardware is easy; the greatest barrier and cost is the process of installing and configuring the software. That's one place a virtual machine is worth considering in producation systems. You aren't going to use something like VMWare in one-of-a-kind production systems. You're going to use it when you need to clone the same set up over and over again. This is very attractive for application vendors, who spend huge amounts of support on installation and tracking down compatibilty conflicts.

Another application would be an IT department that has to support dozens of more or less identical servers, especially if they are frequently called upon to set up new servers. If I had a choice, I'd use Linux virtualization on a midrange or mainframe, but if those servers must be Windows servers, then I'd be looking at some kind of cluster with SAN. This is not really my area of expertise, but we're talking high end boxen for x86; if the typical server didn't need 100% of box, then I have three choices: waste server bandwidth (expensive), force groups to share servers (awkward and inflexible; what if I have to separate two groups joined at the hip?), and virtualization.

Naturally if you are virtualizing production servers, you need to scale your hardware recommendation up to account for VM overhead.

What would be very interesting is a study of the bottlenecks. If you are considering a system with certain characteristics (processor/processors, memory, storage/raid etc) and you have X dollars, where is that best spent?