Virtualizing Cuts Web App Performance 43%

czei writes "This just-released research report, Load Testing a Virtual Web Application, looks at the effects of virtualization on a typical ASP Web application, using VMWare on Linux to host a Windows OS and IIS web server. While virtualizing the server made it easier to manage, the number of users the virtualized Web app could handle dropped by 43%. The article also shows interesting graphs of how hyper-threading affected the performance of IIS." The report urges readers to take this research as a data point. No optimization was done on host or guest OS parameters.

Comments

Virtualize this

[Anonymous Coward]

That is all very well, but we all KNOW apps slow down when we run them in a VM. What difference does it make to the average n00b who wants to watch funny videos and check their email? Anyone using computers for serious numbercrunching obviously won't virtualize anyway. No big deal

Re:Virtualize this

[Fordiman]

I do like the idea of a variably sized beowulf cluster running a floating number of package (LAMP) servers. Get more clients? Add more VLAMPs. Things slowing down? Add more hardware.

You still take performance hits, but if you can scale your system by just adding cheap commodity systems, that works. Plug it in, boot it off a CD, and let the Cluster take control.

Re:Virtualize this

[neerolyte]

The DET (Department of Education and Training in ACT, Australia) struck some deal up with MS to let them run a number of 2003 licences on one box for the cost of one (I have no link, I was working in a school at the time and was actually talking to people who were managing this, not just reading it off the web). Their plan was to virtualize in just this way... you want DNS? Ok that's one VM. What about AD? Ok another...etc etc. I think 7 was about the magic number. I don't know how far they got with this (I've quit since). It may be obvious to the average n00b but unfortunately the average n00b is smarter than many people working for government organisations.

Re:Virtualize this

[eno2001]

Not to mention the fact that paravirtualization as well as hardware assisted virtualization like Xen offers (and later Longhorn) really cut the performance issues WAY the hell down. With a system like Xen you get very close to bare metal speeds since there is no such thing as a "host OS" to get in the way.

Re:Virtualize this

[certain death]

Yeah, that would explain why IBM does it on Big Iron...No one with a Main Frame is doing any kind of serious work.

Re:Virtualize this

[0racle]

The DET (Department of Education and Training in ACT, Australia) struck some deal up with MS to let them run a number of 2003 licences on one box for the cost of one (I have no link, I was working in a school at the time and was actually talking to people who were managing this, not just reading it off the web). Their plan was to virtualize in just this way

Everyone 'struck' this deal. http://www.microsoft.com/presspass/features/2005/o ct05/10-10virtualizationlicensing.mspx

Re:Virtualize this

[sisimon]

This raise some serious concerns on using VM in a production environment.

1. VM is very useful in development environment for making different setup in the testing phase.
2. It is useful in some areas like kernel mode debugging(I use WinDBG in Windows and it is very easy to debug).
3. What happen if the VM kernel or the underlying hardware face some issue ?(from some usual errors like network connection to hardware failure) . All VMs running in the host will die. It even reduce the availability of the entire servers. What is the advantage of having a VM in this case ? Money ?, Availability ?.
4. The performance and manageability, how many customer ready to give-up the better performance for better manageability ?.
5. How many times, customer move to Linux from windows or to Windows from linux so that it need to reinstall the OS in the same hardware ?.

Re:Virtualize this

[Anonymous Coward]

When first read your post i thought you meant a floating point number of package (LAMP) servers. Now that would have been interesting!

Re:Virtualize this

[Alioth]

Actually, you've got it the wrong way around. The Xen authors did a study, and found that virtualization (either User Mode Linux, VMware or Xen) had very little impact on the performance when number crunching. VMware and UML had a very high performance penalty for I/O heavy workloads - i.e. funny videos and checking email. So yes, it will effect the noob the most if the server they are going to is using virtualization, because virtualization has a high performance impact on the sorts of things they do.

Paravirtualization has much less impact on heavy I/O workloads (not all virtualization si equal).

Re:Virtualize this

[That's Unpossible!]

Do you recognize that there are millions of people using computers for something in between "watching funny videos" and "serious numbercrunching"?

Check out Amazon EC2 and tell me again who is and isn't going to be using virtualization?

Re:Virtualize this

[Fordiman]

That's probably entirely possible, too; by resource throttling individual VLAMPs, you could probably set each one up so that it's any fraction of a 'standard server's worth of capacity, or even several.

Well,

[Fordiman]

Duh.

Seriously. I don't know who gave anyone the impression that virtualization was a performance booster. Management improver? Sure. Stability insurance? Why not? But if you don't get that virtualizing your servers imposes a bit of overhead, then you're probably not paying attention.

I especially love the idea that running different types of server virtualized on the same machine is a good idea; the idea of virtualization of multiple servers is to distribute the load. If you have, say, ftpd, httpd and mysqld running as their own virtualized systems, they will all get hit *simultaneously*.

Again. Duh.

Re:Well,

[Mr. Underbridge]

Seriously. I don't know who gave anyone the impression that virtualization was a performance booster. Management improver? Sure. Stability insurance? Why not? But if you don't get that virtualizing your servers imposes a bit of overhead, then you're probably not paying attention.

Well, I think the point was that he attached an actual number to the amount of the performance hit, which is relevant. That's called research; quantifying and proving that which seems 'obvious'.

Re:Well,

[Fordiman]

Well put. But I do know a number of people in the industry that will be shocked by this, which was who I was referencing.

But really. If you've got the money for the extra hardware to maintain performance, I say go for the virtualization, if only to make yout IT guys' lives easier (happy IT is useful IT).

Re:Well,

[hey!]

Well, it's not a surprise, but it's probably worth quantifying.

Here's a question: what is more available: hardware or skilled system administrators? Obviously hardware.

Here's a common scenario: you've set up a system to provide some useful package of services. How do you let other people duplicate your success? (1) tell them what hardware they need and (2) have them install and configurethe software on their hardware. Guess which item involves the most cost in the long run?

The hardware is easy; the greatest barrier and cost is the process of installing and configuring the software. That's one place a virtual machine is worth considering in producation systems. You aren't going to use something like VMWare in one-of-a-kind production systems. You're going to use it when you need to clone the same set up over and over again. This is very attractive for application vendors, who spend huge amounts of support on installation and tracking down compatibilty conflicts.

Another application would be an IT department that has to support dozens of more or less identical servers, especially if they are frequently called upon to set up new servers. If I had a choice, I'd use Linux virtualization on a midrange or mainframe, but if those servers must be Windows servers, then I'd be looking at some kind of cluster with SAN. This is not really my area of expertise, but we're talking high end boxen for x86; if the typical server didn't need 100% of box, then I have three choices: waste server bandwidth (expensive), force groups to share servers (awkward and inflexible; what if I have to separate two groups joined at the hip?), and virtualization.

Naturally if you are virtualizing production servers, you need to scale your hardware recommendation up to account for VM overhead.

What would be very interesting is a study of the bottlenecks. If you are considering a system with certain characteristics (processor/processors, memory, storage/raid etc) and you have X dollars, where is that best spent?

Re:Well,

[Anonymous Coward]

I completely agree with the Duh. But just wanted to add that I have (even very recently) had to talk several of my customers out of doing this on PRODUCTION level high-volume database, application, and web servers! I even had one customer that was insisting that we install all of these machines (total of 6 VM's, I think) onto a single host system using VMWare. They had VMWare reps come in and give them a presentation on how great this would work for them and how performance would not be degraded in any way.

Fortunately, Google Search came to the rescue and I was able to produce benchmarks and reviews such as the one that triggered this discussion. These studies seem like a complete waste of time to those of us that know better, but they can also come in handy...

Re:Well,

[jalefkowit]

if you don't get that virtualizing your servers imposes a bit of overhead, then you're probably not paying attention...

In fairness, a 43% performance hit is a bit more than "a bit". It's cutting performance nearly in half.

I agree with your overall sentiment (a virtualized system is going to be slower than the same system running on real metal, by definition) but 43% is certainly a higher figure than I would have expected.

Re:Well,

[drinkypoo]

To us, the whole point of virtualization is that we have several servers which are mostly idle at all times and completely idle at most times, and for support reasons we're not supposed to be running anything else on the same copy of windows. So we can replace five or six systems with a four-core 1U box with just a few gigabytes of memory, which will recover rack space and reduce power consumption. For anything that's actually heavily loaded, running on the hardware is probably a very good idea.

Re:Well,

[inKubus]

It's not just support reasons. A lot of MSFT products require a dedicated server because they use the Default Web Site in IIS ;) Multi-Tenancy is not an option for many even modern server products. So, virtualize the server.

Re:Well,

[Anonymous Coward]

whoopdifucking dooo... we have one number from one guy about one app... let's all fall to our knees and praise his holy research

Re:Well,

[basil montreal]

That is the primary stated purpose for VMWare. Also, once you are in a VM environment, you can load balance these single ap, low utilization workloads according to their needs. Have a report server that only gets hit heavily once a quarter? Leave it on a slow core for most of the time, and send it to a beefy quad core at end of quarter without having to reboot the machine.

The other factor is the cost of staying current on hardware- if you have 300 single app servers running at 10% CPU utilization and you want to do a mass server upgrade to take advantage of lower power requirements for the newest Xeon processors, running VMWare ESX can allow you to bring the total number of physical servers down by a factor of 5 to 30 (depending on workload).

Everyone knows VMW is not well suited for performance hogging applications. Their whitepapers are fairly clear on this.

Re:Well,

[bean123456789]

It's really not that relevant. Doing a 1-1 comparison of a VM vs the hardware is obviously going to show a performance decrease. But how about a server that runs 5 VMs, 10 VMs, or 20 VMs. What's the degradation there? Do we hold only 43% drop total? Not so bad when you can combine 20 different servers into one solution. A better metric would be to scale the number of VMs to see where the breaking point is, but that is obviously going to depend on what hardware you run it on.

Re:Well,

[don_bear_wilkinson]

The report urges readers to take this research as a data point. No optimization was done on host or guest OS parameters

NO optimization was done

Out of the box, any combo of Windows, IIS and virtualization will SUCK. Got to tweak it. :)

Re:Well,

[fleischdot]

What would be very interesting is a study of the bottlenecks. If you are considering a system with certain characteristics (processor/processors, memory, storage/raid etc) and you have X dollars, where is that best spent? By paravirtualizing production machines, we found the greatest perfomance killer are complex bridging setups with virtual NICs on heavy traffic. This affects not only physical cpu load, more relevant for customers is some kind of 'feeled' performance ;) where network latency caused by dropped packets counts as well. Another bottleneck is of a more physical nature. We noticed, that threaded app's on SMP VM's are remarkably slower if the VM uses cores on different physical cpu's. If vCPU's are pinned to pCPU cores, benchmarks are not very different from physical machines with comparable cpu's. Taking the focus only on virtualization, an overhead produced by a hypervisor is hard to measure. Thus spoken, we're using Xen. And yes, this can't be compared against the article about iis ... we don't have any colorful charts ;)

Re:Well,

[tribentwrks]

Yeah, but you can virtualize MORE machines, and make a cluster of them which could synergistically create a ..... ah, nevermind.

Re:Well,

[performanceengineer]

The article doesn't actually provide that much insight, I think, because the experiment does not really use a realistic scenario for virtualization. It's more common to have larger servers for virtualization for many smaller, lower volume applications. Also, FWIW, I'd like to see more experiments with the Solaris10 Container technology, which looks to me a lot more promising for server consolidation than VMWare.

I already knew that

[Anonymous Coward]

Isn't it obvious? I already knew that performance would drop with Windows OS is hosted on Linux. It isn't WMware fault.

Re:I already knew that

[Anonymous Coward]

You're an idiot

Bogus Test

[Anonymous Coward]

Who uses VMWare Server in a production environment anyway? We run all of our Web services, Exchange servers and SQL databases in VMWare's Virtual Infrastructure 3. VMWare Player and Server are only ment for lab evironments and low load applications. VMWare even says as much on their website. Either this is just FUD or the author is an idiot. In other news water is wet.

Re:Bogus Test

[EraserMouseMan]

So does VMWare's Virtual Infrastructure 3 perform much better? Or is it just more manageable setup and config wise? Sorry, I'm fooling around with VMWare Server and am a noob on the issue.

Re:Bogus Test

[yankeessuck]

The funny thing is that the company I'm working for is in the process of migrating our production internal web servers to VMWare. Our infrastructure director spoke at a meeting of our development group and swore that there's no performance impact He actually said that with a straight face but we all knew he was full of it. I thought about it a bit and decided to let him virtualize my servers because it'll be his job on the line when it tanks, not mine.

Re:Bogus Test

[Sobrique]

Actually, the company I worked for 6 months back, one of the projects I was involved in was 'VMWare'. Production stuff running on on the ESX servers (which became 'virtual infrastructure') in our datacentre, as a cost effective scalable environment. Yes, we weren't getting 'uber performance' but then again, we were running 150 or so VMs on an 6 server VMWare farm.

One of the other things we prototyped and deployed was 'site services packages' - get GSX (now VMWare Server), stick it on a pair of 2U servers, and attach a storage array to both of them. Then create your 'template' fileserver, DHCP server, print server, proxy, that kind of thing and deploy them to this package. It worked very well indeed - you get a whole new order of magnitude on stability (although to be fair that's in part because we through away the crappy workstations that were doing the 'low intensity' stuff) and was extremely managable, and trivially replacable in the event of a hardware failure.

Performance? No, VMWare isn't that great on performance - whilst it's not bad, in an ideal situation, fundamentally what you are doing is introducing an overhead on your system. And probably contention too. But it's really good at efficient resource utilisation, easy manageability and maintainability.

As an experienced sysadmin, my reaction is screw performance. Let's start with reliable and scalable, and then performance just naturally follows, as does a really high grade service.

Proactive laziness is a fundamental of systems admin. Your job, is essentially to put yourself out of a job - or more specificially, free up your time to play with toys. The best way to do this is build something stable, well documented and easily maintainable. Then your day consists of interesting stuff, punctuated by the odd RTFM when something doesn't work quite right.

Re:Bogus Test

[morgan_greywolf]

Actually, I've worked for a place that uses VMWare on some of its production servers. They spend really ridiculous amounts of money on a really big server with a bunch of CPUs (one has at least 96), and then use VMWare ESX Server to run multiple virtual servers on the same box. It's actually a good approach, since ESX server uses hypervisor virtualization, which gives you much lower overhead than traditional virtualization while giving finer-grained control over the resources each virtual server gets.

It's actually not a bad approach, and even works well for servers with a lot of load. I'd like to see *that* technology tested rather than plain-ol' VMWare Server.

Re:Bogus Test

[T-Ranger]

"It must"... VMWare ESX runs on the bare hardware, i.e. no "host" OS. GSX ("Server" now) runs on top of a host OS. Even if it is a very thin, and low overhead layer, the host OS is in the way...

Re:Bogus Test

[tji]

It uses a thin "hypervisor" layer os the Host OS, rather than a general purpose OS. The hypervisor is built to host virtual clients, so I would expect it to provide better performance than doing it on Linux. But, in any virtualization environment, there will surely be some overhead, and the performance will be lower than the client OS on raw hardware.

Re:Bogus Test

[nharmon]

Yes it does. ESX (aka Virtual Infrastructure) the hypervisor (thing that the virtual machines run on) is the base OS rather than an application running on a host OS like with Vmware Server or Workstation/Player. As a result the overhead is extremely low. But I'm not sure that is really relevant. Virtualization is appropriate when your application is not using your iron's full potential. If you virtualize an application that is already maxing our processor/memory, you are not following the published best practices.

Re:Bogus Test

[afidel]

Here's a little spreadsheet I created to do a cost/benefit analysis for Vmware ESX. There are some assumptions built in, and it's not yet a full ROI calculator, but it gets most of the big costs. Cell A1 is the number of our "standard" systems to be compared (4GB dual cpu 2003 machines). The DL580 is 4xXeon 7120 with 32GB of ram, local RAID1 on 15k disks, dual HBA's and a dual port addon NIC. The DL585 is 2xOpteron 8220HE with 32 or 64GB of ram (the 580 with 64GB was more expensive than buying two with 32GB!) and the same equipment. The 360 is our standard build currently, dual 5110's with 4GB ram and local RAID1 and an HBA.

The interesting thing is the breakeven point for VMWare is only 12 servers, way, way below what you can put on two of those boxes. VMotion is the killer feature for me so less than 2 servers is stupid for my situation.

Re:Bogus Test

[sammy baby]

Yes: it performs much, much better.

VI3 is actually a suite of products. At the heart is VMware ESX Server, which is actually an operating system in its own right: it runs "on the metal," without having Windows or Linux installed already on the system. It also has a service console operating system which looks suspiciously like a *NIX style operating system, so you can SSH directly to the system, cd into your /vmfs directory and, say, scp disk files over the network. If you wanted to.

However, as a pretty damn safe rule of thumb, no system is going to run faster on equivalent hardware after being virtualized. In a prior job where I was often asked to provide development/test systems, I got phone calls from a lot of people who were bitten hard by the virtualization bug. Whenever someone brought up any issue having to do with infrastructure, no matter how odd or off the wall, they wanted to push virtualization as a solution. I had to explain to them that if your problem is that a web server is slow, the answer isn't to install VMWare server on it, set up two host operating systems, and say, "There! Now I have two web servers." You'd be surprised how pervasive that sort of thinking is, even among people who should patently know better.

Another useful guideline: various types of services are impacted differently by being virtualized. Generally, the best candidates for virtualization are ones that spend a lot of time idle. This is actually more common than you might think - people need a server set up for something, can't put it on a pre-existing system for security/compatibility reasons, so they go out and buy a new system which is ten times more powerful than they need. You can put a lot of these kinds of systems on a single, reasonably powerful ESX server. On the other hand, systems that heavily tax available hardware, especially I/O, are usually much harder to deal with.

Re:Bogus Test

[PygmySurfer]

At the heart is VMware ESX Server, which is actually an operating system in its own right: it runs "on the metal," without having Windows or Linux installed already on the system. It also has a service console operating system which looks suspiciously like a *NIX style operating system, so you can SSH directly to the system, cd into your /vmfs directory and, say, scp disk files over the network. If you wanted to.

ESX Server is actually based on Linux - it's a very heavily modified version of RedHat. That's why the service console looks suspiciously *NIX-like :)

Re:Bogus Test

[morgan_greywolf]

That's similar to what we found. Note that at 20 servers (10 a piece for 2 of the larger servers), your savings is about 36%. That's huge. One thing I noticed with your spreadsheet, though, is that you don't take into account the extra manpower needed to maintain the hypervisor. We found that we needed one more admin per physical server, and that he/she had to be pretty skilled above and beyond our Windows and UNIX admins (= more $$$) but YMMV.

Re:Bogus Test

[sammy baby]

Ha! You know, I suspected that was the case, but when I did a "mount" command it listed all the partitions like I'm accustomed to seeing them in Solaris, so I wasn't sure. Thanks for clearing that up. :)

Re:Bogus Test

[rolfc]

ESX is lean Linux, Not only linux, but linux.

Re:Bogus Test

[Anonymous Coward]

Generally, the best candidates for virtualization are ones that spend a lot of time idle. This is actually more common than you might think - people need a server set up for something, can't put it on a pre-existing system for security/compatibility reasons Or redundancy/service separation reasons. For example, if you have three data centers and want to spread hosts across all three, you might well want to have six service hosts (so there is redundancy both within and across data centers). Depending on the importance of the service, you might do so with something that could run on a single host (in terms of load).

Similarly, it can be bad to run multiple services on the same host. You can end up with situations where (for example) one service starts spamming its logs and runs out of disk. Do you really want this kind of behavior to bring down other services? There's some vulnerability to bringing the entire real server down in a virtual, but it's much easier to mix and match service sharing with virtual hosts.

Re:Bogus Test

[OldeTimeGeek]

We are.

We have hundreds of servers, most of which aren't even coming close to suffering any performance problems due to over utilization, so it it turns out to be more cost effective for us to use virtualization on systems that are running applications that aren't necessarily processor-intensive (which, in reality, makes almost no difference) rather than the cost hit of hardware and maintenance costs on a bunch of servers that are being underused.

We don't use it for everything, just where it makes sense.

Re:Bogus Test

[ComputerSlicer23]

However, as a pretty damn safe rule of thumb, no system is going to run faster on equivalent hardware after being virtualized.

Oddly enough, that's not as true as you'd think. I know that HP and IBM both had projects to virtual hardware (not in exactly VMWare style virtualization, more like a JIT optimization) where software ran faster after being virtualized. By about 20% if I remember correctly. HP's project was Dynamo and IBM's was DAISY.

Virtualizing a CPU at runtime in a JIT-optimization fashion can actually lead to a significant speed up. Some of this might have been pulled into CPU's by now, but I'm always surprised that more technology isn't moving this direction. Ultimately optimization at runtime has more information then at compile-time, and thus can apply optimizations not otherwise available.

Kirby

Re:Bogus Test

[Richy_T]

Don't try and scp disk files directly from the server, it won't work due to the special filesystem vmware ESX server uses (and unfortunately, you can't export to stdout or to a network share so be careful when setting up initial partitions). You have to export it first. You may need to import them if you scp them to it too (I haven't had the need to do that so far).

Re:Bogus Test

[afidel]

One admin per physical server?????
Dude, I admin 130+ physical servers and ~700 users myself, having another admin would be a luxury. I'm planning on going this way to reduce MY overhead, it's a lot easier to roll out a couple new VM's than it is to rack and stack the equivalent number of pizza boxes =) I just have to make the business case. Having an admin per 20 servers would be purely insane, that would make labor one of the most expensive pieces of the IT budget, which just is not the case around here even if we are well compensated (industry average in a part of the country with about 85% of average cost of living).

Re:Bogus Test

[moshennik]

ESX Server has a Linux kernel... it was originally based on Redhat 6.0 and since was upgraded a few times. It has a bunch of VMware specific commands/utilities, some of them pretty cool (vmware clustered file system for example). We have not benchmarked native vs. virtual with ESX, but it would be an interesting test to try.

Re:Bogus Test

[Bohiti]

Actually, as it's been explained to me, the ESX hypervisor itself is pure proprietary code (and small, too). The Service Console is very readily admitted to be a tweaked out RHEL (3, I believe..). Linux is used to boot, and then (magically?) transfers control of the bare metal to the hypervisor. Linux then jumps into a virtual machine, although it's not presented like a virtual machine, which creates all this confusion.

In the end, the tweaked RHEL that you interact with (ssh, scp) is not the hypervisor, but a VM with special tools that can manipulate the hypervisor.

Re:Bogus Test

[Spectra72]

No you don't. I host templates or golden copies of my VMs on a NFS filesystem all the time. If I'm too lazy to set up the proper vmkernel to mount NFS shares directly to ESX, I mount the NFS filesystem to a solaris box and scp them up.

Re:Bogus Test

[kscguru]

Not really - HP and IBM's project get 20% improvements by optimizing slow code - that is, untuned userspace applications. Take a whole system, including a kernel that multitudes of people have spent years tuning (Linux, Windows), server apps that already squeeze in as many tricks as possible (Apache), and the net gains of re-translating instructions diminish as the underlying apps already pull in more of these optimizations. Dynamo and DAISY also gloss over one crucial detail: you need a good-sized cache to store all these translations, so they're really a time/space tradeoff (more efficient in time with a cost in space efficiency).

That said, the COST of binary translation is never very high. A good BT engine gets essentially native performance (1% overhead is quite obtainable), and is limited only by the size of the translation cache.

Re:Bogus Test

[EtherMonkey]

I had to explain to them that if your problem is that a web server is slow, the answer isn't to install VMWare server on it, set up two host operating systems, and say, "There! Now I have two web servers."

Better yet, I'll setup one $50k 4x Quad-Core Xeon system with 8GB RAM, 4 NIC's, etc and load up $20k of VMWare ESX and I can easily run 8 Web server instances and get ALMOST the same performance and availability as 8 separate $5k servers. Maybe this makes sense somewhere out 3-4 years based on mangement, power and space costs-savings, but I can't see it.

There are right and wrong applications for virtualization. Don't try to force round pegs into square holes.

Re:Bogus Test

[inKubus]

DB Server, no
Dedicated Time Server, yes
Web Front End, maybe

Re:Bogus Test

[CommandNotFound]

Better yet, I'll setup one $50k 4x Quad-Core Xeon system with 8GB RAM, 4 NIC's, etc and load up $20k of VMWare ESX and I can easily run 8 Web server instances and get ALMOST the same performance and availability as 8 separate $5k servers. Maybe this makes sense somewhere out 3-4 years based on mangement, power and space costs-savings, but I can't see it.

In this example, while the single big box might have a lower total peak performance if all 8 VMs were maxed at the same time, this may rarely happen all at once. If you split them up into little boxes, each particular server is limited to be as fast as its individual box. On the big machine, if 7 VMs are relatively idle and one VM has a huge spike, it has 16 cores at its disposal for that time, whereas it would have been limited to the single proc before. Same principle as the mainframe concept, and its usefulness depends on processes idling most of the time, with large sporadic spikes.

Re:Bogus Test

[Quikah]

You don't virtualize 1:1, if your system needs a whole box to do its job it shouldn't be virtualized. That 4x quad core server should be able to handle 20+ VMs if you put enough memory in it. Everything doesn't need 100% of a cpu core 100% of the time, that is the whole point of virtualizing.

Re:Bogus Test

[Anonymous Coward]

No its not. It uses Linux as a console but it is an OS in its own right.

Re:Bogus Test

[dropadrop]

The Service Console is based on RedHat 7 to be precise.

I use and have used both VmWare Server (in the test) and ESX server and they can't really be compared at all. The biggest difference in performance has to do with I/O, both network and disk. While VmWare Server only has around 10-15% overhead in CPU performance, I/O performance is significantly worse... 43% I guess :D

ESX has worse I/O too, but I'd put it somewhere around 15-20% lower then the host. It also depends on how many processors you allocate, virtual-smp is not quite as efficient as it would be. Something which shows fairly well the use of resources is that you can run double the amount of virtuals on the same machine on ESX vs. VmWare Server on RHEL.

But if you look at VmWares recomendations, they don't consider it to be a good idea to make a virtual machine out of a heavily taxed webserver. They mention light to medium web-servers to work well under virtualisation, and this has been my finding too. The whole idea is to take a lot of servers with different kinds of resource needs (and times when they are needed) and share the hosts resources between them. If you take a service which taxes the system to a full there is nothing left for the other virtuals.

You can easily run 15 or more Production server deployments in one 4-core server if you have sufficient memory, and that is where the strengh of the technology lies.

Re:Bogus Test

[dobedobedew]

I just have to reply to this. The product known as "VMWare Server" used to be known as "VMWare GSX Server".
It was approx. $1500 US for a 2-cpu license. IBM, HP, Dell and other OEMs sold it packaged with their servers. It definitely was not meant only for testing.
Since renaming the product and making it free, VMWare has exclusively pushed their VI3 (ESX) product and acted like GSX never was a sold product.

Re:Bogus Test

[Bishop]

ESX is not based on Linux. ESX uses a modified version of Redhat for the system console. If ESX was based on Linux we would have the source.

Re:Bogus Test

[Macka]

This is really interesting for me. I didn't realise until reading this thread that the ESX hypervisor worked that way. Do you know what its capabilities are for (disk) volume management? For instance one solution I'm deploying at the moment for a customer is an HP-UX Campus Cluster, where the SAN storage and cluster stretches over two data centers on the same campus. The volumes are mirrored across both data centers using LVM, so loss of an entire data center doesn't take down the storage or the cluster. For future solutions I've been considering Xen VMs on RHEL 5 and I could potentially support that environment with similar disaster tolerant stretched clusters using CLVM.

Both of those are viable solutions because the OS on the metal can do RAID volume management. Do you know if ESX can too ?

Re:Bogus Test

[cookd]

Nit-pick: There are some nice administrative advantages to VM, so this isn't always true. If the administrative advantages outweigh the overhead, one VM on a host could still be worthwhile.

Re:Bogus Test

[Anonymous Coward]

You guys are blowing my mind!!!

I had no idea it worked that way either, i'm just a lowely programming slaving over hundreds of thousands of lines of code a day now i truely know how sucky my IT hardware department is (friggn main db with one file on one drive and ask me to fix performance).

So i want that and your SAN setup in my apartment anyone want to give me a cost estimate?

Not a trusted source

[Zebra_X]

"Virtualized machine (hyperthreading disabled)
Disabling hyperthreading had negligible effect on the virtualized machine. In our test, the capacity increased a tiny amount to 403 simultaneous users. The difference between this result and the virtualized machine with hyperthreading enabled, however, is smaller than the margin of error for these tests -- more testing would be required before concluding the performance was better in the virtualized machine with hyperthreading disabled. "

The hyperthreaded capacity was 350. So my question is how is a 15% gain in clients served a "tiny amount"?

Re:Not a trusted source

[dagenum]

The hyperthreaded capacity was actually 390 so a 3% gain.

Re:Not a trusted source

[maxwell demon]

Since it's smaller than their error margin, not claiming it to be a tiny amount would mean to admit that there were large measurement errors. But in any case, being smaller than the error margin means that comparing the numbers is completely meaningless; in reality it might not have had any effect, or it might even have been a loss instead of a gain.

Re:Not a trusted source

[basneder]

The hyperthreaded capacity was 350
Only it wasnt, it was 390.

Re:Not a trusted source

[neongrau]

it was the font size that was so tiny you couldn't even read the numbers correctly.

not your fault! ;)

Re:Not a trusted source

[Zebra_X]

Mod me down!

It's a little early here and my vision was a bit blurry, it is 390 not 350!

This is VMware Server and not ESX Server

[Fuyu]

They performed the test on VMware Server not VMware ESX Server which is what most enterprises will use. VMware ESX Server runs on "bare metal", so it does not have the overhead of the host operating system.

Re:This is VMware Server and not ESX Server

[dc29A]

They performed the test on VMware Server not VMware ESX Server which is what most enterprises will use. VMware ESX Server runs on "bare metal", so it does not have the overhead of the host operating system.

Doesn't VMWare ESX run on some modified Red Hat version?

Also, we run ESX in our production environment, when we stress tested a web application running on IIS and with ASP/VB, the ESX machine couldn't give us more than 10 transactions per second (there was one single VM running on ESX). ESX was crawling.

The same hardware running on Windows 2003 native gave us an easy 100+ without any problems. It seems that the overhead of ESX combined with huge number of context switches is what kills the performance of ESX. For non web applications like file servers, administrator consoles and whatnot, ESX is a beauty and great money saver.

For web applications, I would avoid ESX like the black plague.

Re:This is VMware Server and not ESX Server

[Anonymous Coward]

VMWare ESX runs on top of a modified Linux kernel with some VMWare specific modules and drivers for things like VMFS. This striped-down Linux does have a much lower overhead than running something like VMWare Server on top of a bog standard RedHat ES installation, but it isn't "bare metal".

Re:This is VMware Server and not ESX Server

[Fuyu]

Yes VMware ESX Server runs a modified version of Red Hat Linux.

According to Wikipedia, "VMware ESX Server uses a stripped-down proprietary kernel (derived from work done on Stanford University's SimOS) that replaces the Linux kernel after hardware initialization. The Service Console (also known as "COS" or as "vmnix") for ESX Server 2.x derives from a modified version of Red Hat Linux 7.2. (The Service Console for ESX Server 3.x derived from a modified version of Red Hat Enterprise Linux 3.) In general, this Service Console acts as a boot-loader for the vmkernel and provides management interfaces (CLI, webpage MUI, Remote Console). This VMware ESX hypervisor virtualization approach provides lower overhead and better control and granularity for allocating resources (CPU-time, disk-bandwidth, network-bandwidth, memory-utilization) to virtual machines. It also increases security, thus positioning VMware ESX as an enterprise-grade product."

Re:This is VMware Server and not ESX Server

[Sobrique]

I've seen the same thing - just doesn't like context switches. Also IO intensive stuff, tends to start hurting too. But then it's not like I only have one tool in my toolkit :). ESX is a tool for a job, and IMO very good at it. It's not the magic bullet that lets you do anything though.

Re:This is VMware Server and not ESX Server

[Anonymous Coward]

ESX Server still gives you a base 40% performance hit. I run a ~600 VM farm under VI3 and our performance on Apache fell from 15000 requests/s (mostly static content) to 5000. That was during a load test with one single virtual machine running on the blade. The same load test using IIS went from 13000 to 9000. Also a huge performance hit, although not quite as bad as on Linux. And before anyone says anything, I'm a linux tech and I was somewhat deprssed about the results, to our windows techs great joy.

Re:This is VMware Server and not ESX Server

[Quikah]

I think everyone is kind of looking at this the wrong way. Sure you get a performance hit, but yu are testing maximum performance. That is a situation where you wouldn't want to virtualize anyway. If the system is running at 100% utilization, then leave it alone. It is more interesting to take your servers running at 20-30% util (if that, how many idle server do YOU have?), and cram them all into a couple of boxes. You most likely WON'T see a perfromance drop because there was so much headroom on the system already. Virtualization struggles at the max utilization case, but then that is not the case that it is really meant for.

Re:This is VMware Server and not ESX Server

[RedWizzard]

Yes VMware ESX Server runs a modified version of Red Hat Linux.

"VMware ESX Server uses a stripped-down proprietary kernel (derived from work done on Stanford University's SimOS) that replaces the Linux kernel after hardware initialization"

That's not really "running Linux" is it? It boots Linux and then switches to a different kernel.

VMware Server 1.0.1???

[slapyslapslap]

Come on. VMware has come a LONG way. If they are not using at least VMware ESX 3, then this is not a valid test.

Re:VMware Server 1.0.1???

[Fuyu]

VMware Server 1.0.1 is their free virtualization product that runs on a host OS (linux or Windows). Most enterprises will use VMware ESX Server 3 with the VMware Virtual Infrastructure 3 series of products as it runs on "bare metal" and does not have the overhead of the host OS.

This has been my experience too

[SCHecklerX]

Linux under VMWare's network performance is pretty bad. An interesting visual confirmation is to use an ssh shell and watch the lag. That may just be the broadcom chips in the servers the company I was working for used, though. Guest OSes are fine for some low traffic stuff that only a few people will be using, and is definitely the way to go in the test lab; but I wouldn't use this configuration as a company's primary reverse proxy or mail solution.

That said,
I use a windows vmware session under linux for those times I have no choice, and it works just fine network-wise as a workstation.

Re:This has been my experience too

[Anonymous Coward]

If eth0 is shared between host and guest OS and host OS is Linux:

# ethtool -K eth0 tso off

Re:This has been my experience too

[div_2n]

I don't see this at all running 13 VMs. But then again, I've got 6 Gigabit NICS load balanced on a Gigabit backplane with the VMs all running on an independent SAS array on a quad processor hyperthreaded box with 32 GB of RAM. But perhaps your box has equally as good specs, I don't know.

Re:This has been my experience too

[leuk_he]

Linux under VMWare's network performance is pretty bad.

Do you want them to test vista performance?

Re:This has been my experience too

[Thundersnatch]

But then again, I've got 6 Gigabit NICS load balanced on a Gigabit backplane with the VMs all running on an independent SAS array on a quad processor hyperthreaded box with 32 GB of RAM. But perhaps your box has equally as good specs, I don't know.

Oh yeah? Well my Johnson is longer than yours, and my son can beat up your son.

Re:This has been my experience too

[phlegmgem]

...and is definitely the way to go in the test lab... If you are trying to tweak performance, having shared resources can really confuse matters. While trying to optimize a SQL statment on a shared host, I've shot myself in the foot thinking that I was making things better. In reality, performance suffered more due to unrelated system load than from my query stategy. If you have control over all apps that are loading up your physical server, then it shouldn't be a problem. If you're on a publicly shared VPS, then be wary.

Re:This has been my experience too

[Anonymous Coward]

Yes! It is a bug with the broadcom NIC's (other NIC's have the same issue). ESX 2.5.x and ESX 3.0.x have this issue. It is well known at VMWare. A fix should be released in April 2007.

The problem does NOT exist on Windows (except if you have the Windows guest running with other guests which are Linux on the same ESX server).

Is this news to anyone?

[Anonymous Coward]

And with quad core processors already here, do we care? There are major advantages in virtulizing a web application server, like being able to copy your disk image from staging to $n production servers for load balancing.

Sounds about right

[Anonymous Coward]

My first attempt at virtualization was last September with VMWare Server. During testing everything seemed fine. When everything was using it, performance was awful. Everything crawled. I ended up doing an all-nighter to move everything back to a regular server. Note, I wasn't overloading things. There was only one VM on the host. The memory was fixed, not paged to a disk like it is by default. The hard drive was preallocated. My intention for virtualization was to make things easier to manage.

That's when I started experimenting with Xen. This time I put the test under a very high load, and it seemed to handle everything well. I deployed it in October and so far there hasn't been a single performance issue.

I'm now totally addicted to Xen. I create Vms all the time, have split up services into different VMs (ie, when cups crashes it no longer takes out the copy of samba that handles logins, damn I hate cups). So far, no performance issues at all.

Re:Sounds about right

[Alioth]

Yes, Xen rocks. The difference is Xen is a lightweight hypervisor, and VMware Server is heavyweight, more like an emulator. I/O intensive loads in particularly suffer extremely badly under VMware Server (and User Mode Linux), but run almost as fast (i.e indistinguishable) as on the bare metal with Xen.

single data point is correct

[Visaris]

Dell Poweredge SC1420 with dual Xeon 2.8GHz processors

While I can't seem to find all the information on the SC1420, it appears as though this product uses processors from the Prescott generation of Intel CPUs. Some chips from this group support "Vanderpool", Intel's hardware virtualization solution, but not all do. The presence or absence of this feature could greatly impact the performance penalty faced by operating a virtualized computing environment. Further, Intel's new Core2 based CPUs feature a hardware virtualization implementation which may have vastly different performance characteristics. AMD's K8 family supports hardware virtualization as well. I'm excited about their new line of CPUs based on the K10 (Barcelona) core, which feature "NestedPageTables," which are supposed to greatly reduce overhead by doing memory translations in hardware instead of in software by the hypervisor.

All I'm really trying to say is that this article really is only a single data point. I wouldn't let their results influence your overall view of virtualization in any way...

Re:single data point is correct

[crunchy_one]

Absolutely correct. The presence of virtualization hardware, specifically how well it handles address translation, is key to virtual machine performance. Intel and AMD have just begun on this path, so I would not expect to see near-native performance out of their virtualization hardware for at least the next two or three iterations. IBM followed this path with interpretive execution in the 80's and 90's. It took several iterations of the hardware/software combination before interpretive execution and VM/XA delivered near-native performance.

Re:single data point is correct

[crunchy_one]

While I'm blithering away on this topic, I'd like to point out that the x86 architecture makes no attempt to specify how I/O devices access memory. This is a huge open problem for virtualization performance and security. To be secure a virtual machine monitor must simulate all I/O; otherwise, an attacker in one virtual machine can access any memory belonging to another virtual machine through I/O direct memory access. Ignoring the security issue, the lack of a uniform architecture for I/O memory access makes virtualization difficult, leading to sub-optimal solutions such as having all I/O performed by one authorized virtual machine.

Re:single data point is correct

[TheRaven64]

The biggest overhead from most forms of virtualisation is from emulated devices. If you have loads of money, you can give it to IBM and get some hardware with virtualisation-aware network and block device controllers. Then you get good performance. Alternatively, you can use paravirtualised device drivers. Xen supports this by default, and I think KVM does now for networks. Not sure about VMWare.

With paravirtualised devices, or devices that are virtualisation-aware, a VM can be within 10% of the performance of a real machine quite easily. Without I'm surprised they even got to 57% of native performance for web applications.

Re:single data point is correct

[Natales]

VMware's vmxnet driver is paravirtualized and it does provide better performance than the traditional pcnet32 virtual device driver, which operates 100% on software to maintain compatibility with other OSs.

Regarding paravirtualization, it's already known that the new VMware Workstation 6 (currently in beta) and presumably the next version of VMware Server, will support VMware's version of paravirtualization called VMI, which was officially accepted as part of the stock Linux kernel starting on 2.6.21. This may help boosting the performance of Linux-based VMs significantly, and unlike the Xen version, it will boot a single kernel image, regardless of the physical or virtual underlying hardware platform.

Re:single data point is correct

[TheRaven64]

unlike the Xen version, it will boot a single kernel image, regardless of the physical or virtual underlying hardware platform I believe your information regarding Xen is out-of-date. The current XenoLinux uses a jump table for privileged operations that issues Xen hypercalls or native x86 operations depending on which is detected at boot. Oh, and Xen can already run unmodified Linux (or Windows) with paravirtualised drivers on platforms with hardware support.

Pointless test?

[geoff lane]

Come on! You run virtualised web servers because 99.9% of all web servers are idle at any given time. So you put 100 on a server. The customer doesn't see any worse performance with their 3 hits a week page and the ISP makes more money/server.

Re:Pointless test?

[Software]

Did you know that one installation of Apache can serve multiple web sites? IIS can do the same. Using 100 guest OSes running on a server to support 100 web sites is insane.

Re:Pointless test?

[jimicus]

True in some businesses.

But if you're speccing up a web application that you can be fairly certain will be used by hundreds of people simultaneously, then it's useful to know.

Of course, if you're speccing up the system that this web app is going to run under and you don't test performance before you go live, you'll come unstuck sooner or later anyhow.

Re:Pointless test?

[TheRaven64]

Virtualisation gives you one major advantage over this; isolation. I have a single isntance of Lighttpd serving a few web sites, but if I chose to be malicious then I could easily put something in one that would seriously degrade the others. If each site had a separate server, then this would not be possible. Each site could be owned by a different person / organisation, and completely isolated from the others.

Re:Pointless test?

[Anonymous Coward]

Did you know that one installation of Apache can serve multiple web sites? IIS can do the same. Using 100 guest OSes running on a server to support 100 web sites is insane.

Unless the customer really, really wants to have their own OS image with complete control of everything including root access.

Re:Pointless test?

[Bert64]

Some customers don't like the idea of doing that...
Also, Apache normally runs all the sites as the same user, which is terrible from a security perspective. There are alternatives here, but all have their downsides.

Re:Pointless test?

[LurkerXXX]

No it's not insane. Lots of customers want full root access on their systems so they can install whatever they want (different database or other servers, or even alternate OS's). Virtualization is the only way to go for that.

Re:Pointless test?

[jafiwam]

True for "mom and pop" type and "I need a web site for my dog or clan" sites.

Once you start in the market of "32 year old hotshot ASP/PERL program guy that never ran is own server" territory (medium to small businesses have these all over) who has no oversight from his management team... you tend to get people who accidentally kill the entire OS by doing stupid shit in their web sites. "Let's just build an app that sends email every time a file gets viewed! Then I can build my log in Excel and see all my hits real time!" (Never mind that there are logs available to download real time.) Of course, this tool doesn't realize that that crappy DreamWeaver menu he built contains 125 images which also count as hits. Down goes the server and down (until the web server stops) goes the mail server.

People who host on other servers have no respect for their integrity even if they know how. Like a public bathroom, who cares if the toilet is plugged, just shit on top and walk away.

If you want to host with those people and not go out of businesses, you gotta virtualize to protect the rest of your customers from the overblown wannabe leet geeks. The, "knows enough to be dangerous" crowd. Either that, or you need a TOS so strict they end up going elsewhere (or self-hosting).

So, yeah, IIS and Apache and whatever can host many web sites with one instance. Sometimes though, you can't realistically do that.

Re:Pointless test?

[petermgreen]

Did you know that one installation of Apache can serve multiple web sites? IIS can do the same.
indeed it can

now add on the fact that you need mail for each domain stored in a different place (generaly people want e-mail on thier websites domain).

now add on the fact that you have to be very carefull to stop active content doing nasty things to others sites.

then you have the issue that not everything about apache can be controlled through htacess files

it gets even more complex if you have multiple users per site.

now there are frontends that try and deal with theese admin problems for you (cpanel, webappliance etc) but they have issues of thier own (they tend to get in the way if you want to have services other than those thier designers thought of).

Using 100 guest OSes running on a server to support 100 web sites is insane.
depends on the situation. If all sites are run by one admin team then sure just configure the stuff you need to handle multiple domains. If every domain just needs simple webhosting with basic script requirements then sure go ahead and use cpanel or webappliance or similar. If each domain has some scripts with strange requirements or requires other low load services in addition to web and mail or you want complete independence in management then virtual machines start to shine.

Re:Pointless test?

[Albanach]

It's not insane if people want different solutions or even want their own server. With virtualisation, a host can offer multiple php versions. You can avoid all the security problems where one script running as the webserver can read any other file accessible to the web server.

You can also get better management control of resources, preventing one site from eating up all available resources on the box.

That's not to say there aren't a million good reasons to use virtual servers in apache, just to point out that virtualising web hosts is not, by definition, a daft idea.

Re:Pointless test?

[afidel]

Except you can offer better security and more flexibility with virtual servers than traditional shared hosting. For a site with no DB or custom native code then shared instances is fine, for anything requiring code running on the server virtual servers are better. The cool thing about ESX is if your servers are all running the same OS and version of their apps the static memory contents all get laid onto the same memory pages, meaning that increased ram usage for 20 servers vs 10 is very low.

Re:Pointless test?

[fatphil]

An Apache, just do:
        AssignUserID user-id group-id
in each virtual host definition.

It really couldn't be simpler.

Re:Pointless test?

[GiMP]

AssignUserId only works with the perchild MPM, which has the following caveat: "This module is not functional. Development of this module is not complete and is not currently active. Do not use perchild unless you are a programmer willing to help fix it."

Thus, AssignUserId should NOT be used. SuExec can be used, of course, but that has its own limitations.

Personally, I give users their own Apache processes on their own port (>1024) and use a reverse proxy. I make a living on it.

Re:Pointless test?

[Just Some Guy]

You run virtualised web servers because 99.9% of all web servers are idle at any given time. So you put 100 on a server.

If you have a real need to run 100 separate Apache instances, then you'll want something much higher-level than VMWare. For us, that would be a FreeBSD jail, where each instance would get its own chrooted home directory and IP address. That way, you're not allocating resources to 100 little-used OS images; each shares from the same memory and hard drive pool. Jails are slightly limited in that I'd like a way to limit CPU and memory allocation, but in practical application this really works very well today.

Re:Pointless test?

[fatphil]

Thanks for that correction. What a bunch of bodgers. I had assumed that as it was in 2.0 but not 1.3 (I only use 1.3), that it was a feature that had been designed in specifically as a required feature. Alas, not. I should R TFM rather than just grepping it.

Re:Pointless test?

[julesh]

If you have a real need to run 100 separate Apache instances, then you'll want something much higher-level than VMWare. For us, that would be a FreeBSD jail, where each instance would get its own chrooted home directory and IP address. That way, you're not allocating resources to 100 little-used OS images; each shares from the same memory and hard drive pool. Jails are slightly limited in that I'd like a way to limit CPU and memory allocation, but in practical application this really works very well today.

Speaking as a former user of such an application and current user of an equivalent virtualised system, the latter works much better. The chroot-jail based hosting system I used had a number of problematic limitations: I couldn't install any software that needed to run as root (e.g. to open ports below 1024), I couldn't change configuration of a variety of services that would open security problems for the hosting provider (e.g. the mail server, I think they were using exim), users running processes on the same server that consumed too much CPU and or memory had too much negative impact on my server's performance. I switched to a provider that uses virtualization, and all of these issues are gone.

Speaking as an administrator of virtual servers, I'd like to point out that one of your assumptions is wrong: VMWare ESX server *can* share memory and free disk space between instances. Multiple instances can even have the same page of physical memory mapped into them, so running multiple identical servers doesn't actually take more memory than it does on a single instance (although there are overheads).

Re:Pointless test?

[Just Some Guy]

The chroot-jail based hosting system I used had a number of problematic limitations: I couldn't install any software that needed to run as root (e.g. to open ports below 1024), I couldn't change configuration of a variety of services that would open security problems for the hosting provider (e.g. the mail server, I think they were using exim), users running processes on the same server that consumed too much CPU and or memory had too much negative impact on my server's performance.

Of those, only the last is relevant to FreeBSD jail setups. If I created a jail for you and gave you root, you would be root, full stop. The only things you could do would be install your own kernel (since only one kernel - that of the host OS - is running). We use them to virtualize multiple distinct systems on the same hardware, with the idea that the mailserver always runs under a light load and doesn't interfere with the database server hosted on the same machine.

There's a decent Wikipedia article on the subject, even if it kind of comes across like an advertisement. In short, it sounds like your hosting provider ran a bad server. Don't extrapolate their incompetence to the general state of the art.

VMWare ESX server *can* share memory and free disk space between instances. Multiple instances can even have the same page of physical memory mapped into them, so running multiple identical servers doesn't actually take more memory than it does on a single instance (although there are overheads).

Assuming a strategy like copy-on-write, I can understand how two instances started from the same configuration could begin with most of their memory shared. However, it seems like that would eventually become a tiny percentage of their actual address space as processes start and die, allocate memory and free it, etc. I mean, to the best of my knowledge, when Unix fork()s a process, it doesn't keep track of when it can later re-merge the address space of the parent and child. They may start as identical copies, but pretty soon their data segments will be completely different. In the case where each process is actually a virtualized system where the data segment is hugely bigger than the shared code, I'd think that would happen pretty quickly.

Re:Pointless test?

[julesh]

Assuming a strategy like copy-on-write ...

That's not how it works; in unused processor time it scans memory for pages that have identical content and merges them.

Hidden advertisment

[quigonn]

This smells like a hidden advertisment for "Web Performance Inc.". Now somebody please tell me why I should trust the results produced by a relatively unknown product and company, and not stick to proven tools like Borland SilkPerformer or Mercury Loadrunner.

Re:Hidden advertisment

[Monoman]

That and maybe a little bit of badmouthing the free VMWare Server product to drive people to use the MS Virtual Server?

All and all nothing great to read unless you are new to virtualization and some of the basics explained to you.

Virtualization is like any other tool, it isn't the always the answer but when appropriate it is a good thing.

Bad data, bad setup

[duncanFrance]

There's quite a lot wrong with their setup.

1) As others have pointed out, they should be running on ESX to get best performance.
2) Physical machine was a dual-proc. How many processors did they assign to the VM?
3) Physical machine had 2GB memory. They assigned 2GB to the VM!! Vmware will take 256MB of this
for itself, so that 2GB visible to Windows will be being swapped.
4) How many disks did the physical machine have, and what was on them?
If e.g. the physical machine had two disks, the VM should have been given two disk files, with each file being placed on a different physical spindle.

You get the picture.

Re:Bad data, bad setup

[suv4x4]

There's quite a lot wrong with their setup.

1) As others have pointed out, they should be running on ESX to get best performance.
2) Physical machine was a dual-proc. How many processors did they assign to the VM?
3) Physical machine had 2GB memory. They assigned 2GB to the VM!! Vmware will take 256MB of this
for itself, so that 2GB visible to Windows will be being swapped.
4) How many disks did the physical machine have, and what was on them?
If e.g. the physical machine had two disks, the VM should have been given two disk files, with each file being placed on a different physical spindle.

You get the picture.

And still 43% performance drop-off is quite good for the kind of benefits virtualization gives. People pay way heftier performance penalties by using slow language (PHP), frameworks (Ruby On Rails) and plain coding poorly and not caching what is cacheable.

Re:Bad data, bad setup

[linuxgurugamer]

Ummm, you didn't read the article. They specifically said that by assigning 2 gig to the VM that the linux based system actually had more memory.

Re:Bad data, bad setup

[duncanFrance]

I did read the article:

"The virtualized server has the same memory available to it (2G) as the native server (which implies that the physical machine running VMware has more memory)."

It might "imply" it, but they fail to tell us how much memory the physical machine actually did have. Or whether VMware was set up to assign the memory in one block. Coz if not, yes, you guessed it, it's swap time again.

Re:Bad data, bad setup

[j-cloth]

2) Physical machine was a dual-proc. How many processors did they assign to the VM?
Great points, but for this one I must answer the rhetorical question for clarity. The answer should be 1. VM Guests will perform very poorly when given 2 vCPUs on a 2 CPU host because (simply) the guest will not be given access to any CPU until both are available. This is something VMWare says all over the place but it's tough to convince sysadmins who are coming from the physical world that more CPU != faster.

well duh

[Mantaman]

Do i need to say more? We all know running any VM app is always going to be slower than a real OS on real Hardware. Ok mabe I didnt realise how much slowdown their was but there are easier and cheaper ways to run web servers. Nice to have given it a go and gathered stats but nothing new.

Re:well duh

[Sobrique]

Depends on your objective. If you want something that runs the fastest it possibly can, yes you're probably right. Don't use VMware. Regardless of whether it's web, database or ... well whatever, VMWare is an overhead.

In a perfect world, as a sysadmin I create a nice stable system, and it works really well. Maybe I run lots of apache instances on a single machine, and they co-exist happily.

But this isn't an ideal world. What VMWare does, is fundamentally to workaround the fundamental bogosity of Windows. It lets you create an environment that's trivial to replicate, hardware abstracted, and doesn't trash the rest of the machine around it, when 'someone' inevitably screws up. Someone does something stupid with your single instance apache server, and it goes splat as they fill up /, or get a broken CGI on there, or something. Same things happen to a VM, but then it's dead easy to fix.

There's a reason that almost all the hardcore vendors are 'virtualising'. It's because despite the performance overhead, the bonus you gain on managability, scalability, reliablity far outweighs the cost of having to buy a few more boxes. When you're talking about large sums of money riding on 'outage' this is a no brainer.

I'd still run websites on vmware, because then I can give users the power to do what they like, without screwing up my overall service. That's a good trade in my book.

holy cow am I a nerd

[thegnu]

I do like the idea of a variably sized beowulf cluster running a floating number of package (LAMP) servers. Get more clients? Add more VLAMPs. Things slowing down? Add more hardware.

I started getting aroused as I read your post. This is highly disturbing.

Re:holy cow am I a nerd

[Fordiman]

A sister of my department here actually does this for its academic hosting. Though, it's a grid, not a cluster.

Re:holy cow am I a nerd

[Anonymous Coward]

Is your department's sister hot?

Re:holy cow am I a nerd

[Anonymous Coward]

PICS or GTFO

Re:holy cow am I a nerd

[slack_prad]

What.. getting aroused? VERY disturbing.

Re:holy cow am I a nerd

[Anonymous Coward]

TITS or GTFO ^ Fixed

data, datum, data

[frequnkn]

I know that the term 'data' is generally used as a singular in informal speech, but it still drives me nuts. I bet there are a lot of other current or former Latin club members that howl at this literary fingernails-on-chalkboard usage of the term. Not that I stayed after school for Latin club meetings, or to play Civ, or D&D...

Silently weeps into his tattered copy of Remedia Amoris

-Foo

"Duh!" moment

[Thumper_SVX]

I agree with many of the commentators here that this is pretty obvious. We use virtualization a lot, but also realize its limitations. For example, we don't run SQL or anything heavily transaction or I/O bound. CPU utilization is usually not a problem; virtual machines perform as well as their physical counterparts in most instances unless you have a lot of CPU intensive virtual machines running.

Web servers are mostly memory and CPU bound which would give one the impression that they would be great candidates for virtualization. However, VMWare Server is not the solution; network I/O is not good on Server. Typically your results would be maybe 75% of the actual physical speed on a "passthrough", less on a NAT. It depends a lot on how your network is set up, not to mention the abilities of the physical machine.

The best solution is Virtual Infrastructure (used to be ESX). That product tackles most of the failings of VMWare server and fixes them. The only exception is that I still wouldn't run anything I/O heavy on VI. SQL's a no-no. Also, if you're not getting the performance from a single web server that you expect, you can easily throw up more web servers. Now, obviously you might get into M$ licensing issues, but that's why you run your web services on Apache :D

Use Virtuozzo

[pyite69]

Of course VMware and Xen are going to be slow - that is the tradeoff you get when you want the ability to run both Windows and Linux at the same time.

http://openvz.org/ - it does a much better job of virtualizing IMO. The only minus is that all VM's have to use the same kernel version.

Re:Use Virtuozzo

[DaemonTW]

This is what most hosting companies who offer VPS's do, as Virtuozzo is one of the only virtualisation packages that doesn't suffer from massive performance hits. Granted this is more about what level things are virtualised at than better coding but the performance difference is very significant.

I've seen servers running over a hundred virtual servers (mostly all low usage) without any problems, something that is completely out of the question with VMWare (even ESX).

I'm sure there will be many who will point out that it has less isolation and therefore less secure, however so far I haven't heard of any compromises.

Xen

[Verte]

Or better yet, if you need better performance under load, why not Xen? I expected to see something in there on paravirtualisation, but nothing.

How does the webapp scale?

[l0b0]

Unless it scales linearly with the number of users, that is a pretty useless metric for the performance of the virtualization system (No, I didn't RTFA).

Obviated Comment

[ursuspacificus]

I have decided to refrain from commenting on this article, as I do not wish to be tarred with the epithet "me-too-er".

Having said that, "Me, Too!"

Fast Virtualization: Xen, KVM, Virtuozzo, GSX, ESX

[dvdan]

For speed, the newer virtualization tools KVM, Xen, and Virtuozzo are presently substantially ahead of the present incarnation of VMWare. KVM requires the new "hardware virtualization" CPU's from Intel and AMD which must be mentioned here, since they represent a major industry recognition of the value of virtualization. This article seems to be giving people the impression that performance of VMWare Server is indicative of virtualization tools in general, and that all virtualization tools slow down hosted virtual machines dramatically. This is simply false. I know hosting providers running 50 virtual servers on a single dual CPU box with thousands and thousands of users, which would simply not work if all virtualization tools had a 43% hit per instance. Another key matter here is that the author fails to mention (or realize?) VMWare Server is crippleware. VMWare states explicitly not to use VMWare Server for anything other than testing because it does not have the performance or feature set of their full blown ESX and GSX servers. Also, while VMWare may be the oldest and arguably most mature virtualization suite, it is certainly not the fastest.

People still use IIS?

[DragonTHC]

who are these people? why are they not being publicly flogged?

Re:People still use IIS?

[jerw134]

Do you have any legitimate criticism to offer towards IIS, or are you just trolling?

Quantifying

[Vexorian]

I think it was pretty obvious that it would add overhead and therefore drop performance. This study is good for quantifying how much of a performance threat it is. And 43% is not an incredibly bad value in my opinion, it is not even 50% ...

These results are pretty much as expected

[Sangui5]

It isn't surprising that VMWare would be bad at a web-app workload. See the original paper on Xen:

http://www.cl.cam.ac.uk/research/srg/netos/papers/ 2003-xensosp.pdf

Top of page 9 has a chart comparing native Linux, Xen, VMWare, and UML for different workloads. They show VMWare degrading performance by over 70% for SPECWEB 99.

Web applications are OS intensive; while VMWare is quite good at pure CPU-bound tasks, it has to perform a lot of emulation whenever you are running inside the OS. So it will stink at anything with lots of small IO, lots of metadata operations, or lots of process creation/switching. For example, VMWare shows a whopping 90% slowdown for OLTP database workloads, according to the Xen paper, and it really isn't surprising. The OS microbenchmarks in the above paper (page 10) show that VMWare has abysmal performance for things like fork(), exec(), mmap(), page faults, and context switches.

Basically, Xen doesn't have to emulate the OS, because they make modifications to the OS. VMWare does dynamic binary rewriting (think fancy emulation) to run an unmodified OS; they therefore pay through the nose in performance overhead for OS-intensive workloads.

Re:These results are pretty much as expected

[Anonymous Coward]

You reference a 2003 paper which uses SPECWEB99, are you fucking kidding!?! And you say that Xen doesn't have to emulate the OS so its much faster. VMware doesn't emulate the OS, they implement a hypervisor and emulate I/O. VI3 has changed the rules with respect to what can and should be virtualized. Wiped the slate clean. You go from bad to worse, and like everyone else here who uses the words "ESX" or "farms" should be ignored, for you know not that which of you speak.

And yes, I am a large enterprise, multi-platform virtualization architect.

"Typical Application" they for sure ment 42%

[drolli]

What what the typical application. Was it well writen or not. what was the reason for the slowdown (Memory, Network, ?). Without that 43% is just a munber as good as 42%, the answer to all performance loss questions

Xen Scales

[ndverdo]

The CPU-overhead on the Xen hypervisor is much much lower - between 2-4%. http IO has been less explored. There have been workshops on its use in HPC http://xhpc.wu-wien.ac.at/ http://xhpc.ai.wu-wien.ac.at/

Re:Xen Scales

[Anonymous Coward]

As the guy who ran XenSources performance testing, I can confirm this. specjbb workload, non-VT hardware (paravirtualization, in other words, not hardware virtualization), scaling to ~16 guests per system.

The curves were: increasing guest memory over the ~900 MB boundary would degrade performance slightly as the hypervisor was required to double-compute address locations for high memory, when used (32 bit architectures only). And Xen scaled to SMP better than Linux. That is: Running 4 Xen guests on a 4-way system performed better than a single native 4-way instance.

The referenced article's performance characteristics are specific to VMWare. Not all virtualization methods. Xen, Linux Virtual Servers, and Kernel Virtual Machines (KVM) should all scale quite well.

Disclaimer: I'm a former XenSource employee, no longer.

KMSelf

That's easy to fix!

[JoeD]

Since a virtual server only gets 57% of the performance of a physical server, just run TWO virtual servers!

That way, you'll get 114% throughput!

Next week, I solve world hunger, global warming, and bring peace to the Middle East.

IIS can't be paravirtualized

[tepples]

Not to mention the fact that paravirtualization as well as hardware assisted virtualization like Xen offers (and later Longhorn) really cut the performance issues WAY the hell down. Paravirtualization also requires a free software OS kernel, and IIS-only web applications are do not yet run on any free software kernel. (ReactOS is nowhere near mature enough.) Any virtualization also requires more OS licenses and higher-class, more expensive OS licenses. Or do you claim that all web app developers should drop IIS-only frameworks immediately, and all enterprises that rely on IIS-only web applications should drop their mission-critical IIS-only web applications immediately?

Re:IIS can't be paravirtualized

[rolfc]

Nobody say they should drop them immediatly, we are just saying that free software rules and closed source sucks. Then everyone has to draw their own conclusions from that.
I do not understand people that make mission-critical IIS-only webapplications, isnt that just stupid? Smart persons make sure that there are some emergency exit.

Re:IIS can't be paravirtualized

[Joe U]

I do not understand people that make mission-critical IIS-only webapplications, isnt that just stupid?

No.

It's no different from focusing on any other major corporate product. ASP.NET isn't going away anytime soon and it's not going to fall over anytime soon.

It's run by Microsoft. Despite what the ranting morons around here think, Microsoft is not going away, IIS is not an insecure mess and it's actually a really good platform for web applications.

Now, that being said, use the best tool for the job, otherwise you're just stupid.

Re:IIS can't be paravirtualized

[Ryan Amos]

You can yell at me and tell me I'm wrong, but can't you use apache2 instead of IIS, even with ASP.NET? (bear in mind I use neither IIS nor ASP.NET.) I would assume the application is just using the web server to pass packets and manage connections.

Re:IIS can't be paravirtualized

[jopsen]

As far as I know you can run windows under Xen, and other paravirutalized environments, if your CPU has support for paravirtualization technology like AMD's Pacifica and Intels Vanderpool... Source: http://en.wikipedia.org/wiki/Xen#Windows_as_guest/ So using ReactOS shouldn't be necessary, don't think anybody would every use it for important thinks ever. By the way, there's hopefully nobody running anything close to mission critical on ISS/Windows...

Re:IIS can't be paravirtualized

[bertybassett]

I sometimes say "you should drop them immediately" to my woman.....

Re:IIS can't be paravirtualized

[rolfc]

They may not go away tomorrow, but they can be quite expensive, so the problem is that if you have no choice you have no bargainpower. And I dont think it is such a good platform for web applications, and it is more expensive in those cases I know. The arguments I have heard in favour is that there is plenty of cheap consultants to be found and Microsoft is "the" industry standard. My former boss believed that if we run only Microsoft, we will deliver more for less. Well, we didnt.

Re:IIS can't be paravirtualized

[tepples]

As far as I know you can run windows under Xen, and other paravirutalized environments, if your CPU has support for paravirtualization technology like AMD's Pacifica and Intels Vanderpool But does low-end server hardware include the necessary support for AMD-V (Pacifica) or Intel VT (Vanderpool)? And what about the software licensing issues? Do the end user license agreements of Windows Server 2003 and the forthcoming Longhorn Server allow the use of Windows OS inside Xen virtualization?

Re:IIS can't be paravirtualized

[joto]

Paravirtualization also requires a free software OS kernel

Uhm, why? It's not like the virtualization program reads the license to the source code, or anything. I think you are confusing the issues here.

Or do you claim that all web app developers should drop IIS-only frameworks immediately

Yes, that would be a good idea

and all enterprises that rely on IIS-only web applications should drop their mission-critical IIS-only web applications immediately?

Yes, at least as soon as possible.

Re:IIS can't be paravirtualized

[Poltras]

Despite what the ranting morons around here think, Microsoft is not going away, IIS is not an insecure mess and it's actually a really good platform for web applications. YMBNH.

Re:IIS can't be paravirtualized

[tepples]

Paravirtualization also requires a free software OS kernel Uhm, why? Paravirtualization presents a simplified architecture to the guest operating system. This requires that its kernel be modified to run on the simplified architecture. This is not possible in proprietary software distributed under licenses similar to that of Microsoft Windows OS.

It's not like the virtualization program reads the license to the source code, or anything. No, but a judge does. And who outside Microsoft has the Windows source code anyway?

Re:IIS can't be paravirtualized

[joto]

Paravirtualization also requires a free software OS kernel

Uhm, why?

Paravirtualization presents a simplified architecture to the guest operating system. This requires that its kernel be modified to run on the simplified architecture. This is not possible in proprietary software distributed under licenses similar to that of Microsoft Windows OS.

I believe Microsoft does have one or two engineers who would be competent enough to do the job, if they were asked to do it, don't you?

It's not like the virtualization program reads the license to the source code, or anything.

No, but a judge does. And who outside Microsoft has the Windows source code anyway?

Why would a judge be needed for Microsoft to modify their own operating system for paravirtualization?

As I said, you are confusing the issues. Just because you can't run windows through paravirtualization, doesn't mean that it isn't possible. What is needed is either access to source code (and legal rights to modify it), or a cooperative vendor. In particular, free software, is certainly not needed, nor open source. A cooperative vendor, and/or source released under an NDA is perfectly acceptable.

Re:IIS can't be paravirtualized

[jerw134]

can't you use apache2 instead of IIS, even with ASP.NET? No, ASP.NET only runs (in any meaningful way) on IIS.

Re:IIS can't be paravirtualized

[eno2001]

Whoa there boy... you need to learn a little bit about today's virtualization methods. For one, Xen is quite important in this regard since it is entirely possible to paravirtualize Windows. In fact it's so possible that MS modified the Windows kernel to work on Xen, they were so interested in what Xen was doing. Then, following that they cooperated with Xensource to provide a way of reaping the benefits of paravirtualization without need to rewrite the kernel. When you install the commercial version of Xen you get a special MS approved driver pack that sits in your Windows VM Domain to work with Xen and the Pacifica and Vanderpool technologies of AMD and Intel respectively. The performance is again approaching bare metal. Something that VMWare could only do on their higher end products but which is now possible with new servers and Xensource. Stick to what you know... you apparently don't know jack about virtualization. It's very different today from where it was a year or two ago.

And as someone else pointed out, quit trying to muddy the waters by bringing licensing into it. The fact is that When Longhorn comes out, it's supposed to contain a hypervisor in it to do just what Xen has been doing for the past few years. Basically a thin layer on the CPU metal that the OSes ride on. No more "host" OSes to slow things down. But, if you're like me, why wait until MS has a solution out when you could have already been using one for as long as the CPUs with VT/HVM have been available? Of course, if you're like me, you're only running one instance of Windows XP Pro so you can access content that isn't enlightened enough to be multiplatform friendly. The majority of the VMs I run at home and at work are... drumroll please: Linux. Why you may ask? Because it "just works" for me. Windows doesn't.

Re:IIS can't be paravirtualized

[julesh]

Paravirtualization also requires a free software OS kernel, and IIS-only web applications are do not yet run on any free software kernel.

Not strictly true. MS are willing to license their kernel source under a shared source agreement, and the Xen people have done so and have made XP run under paravirtualization. The resulting OS isn't commercially distributable, but that doesn't mean you couldn't license MS's source and Xen's patches to it and get your own paravirtualized Windows kernel.

Any virtualization also requires more OS licenses and higher-class, more expensive OS licenses.

Standard Windows volume licensing terms allow you to run 4 instances of each licensed copy of Windows in VMs if you wish. This is, of course, substantially cheaper than buying 4 licenses to run those instances on separate hardware.

Would be interesting if done right

[ceeam]

1. They fucked up their setup. They assigned 2Gigs to VM and all host has is 2Gigs too? Brilliant.
2. Since when are you allowed to post benchmarks of MS software?

Re:Would be interesting if done right

[merrill77]

No, actually, we didn't f**k up our setup. To quote from the article: "The virtualized server has the same memory available to it (2G) as the native server (which implies that the physical machine running VMware has more memory)." Apparently this sentence is confusing to some, maybe because it was not obvious that we had two completely separate machines in the test setup. I'll try to clarify: When the physical server was tested, it had 2G. When the virtualized server was tested, it had 2G. The physical machine running the virtualized server has _more_ than 2G. We should have stated how much - it had 4G, all of which was available to the host OS. I hope that clears it up.

Re:Would be interesting if done right

[Anonymous Coward]

By the way d00d, the VMware license prohibits posting benchmarking results. This has always been the case and it's the reason why you never see VMware benchmarks.

Prepare for much lawyer speak. Ah hell, they'll probably be nice and just tell you to take it down.

Re:Would be interesting if done right

[kscguru]

No, you did totally screw it up.

• VMware Server vs ESX. One is for development and is free; the other has real performance. Ballpark, Server loses 15-20% of the performance you can get from ESX.
• CPU counts. Host had 2 processors (well, 4 with HT). No idea what your guest had, but I'll assume 2 ... catch is, vSMP guests are experimental in VMware Server: they're known to have a non-trivial performance hit (~15% right there). If you really want to compare, set up TWO VMs, each with one processor, and see what overall performance is. Even at 1GB per VM (which is unfair, BTW - Server can share memory between the two, 1.3GB per VM is more realistic), I bet two 1P VMs will have more throughput than a single 2P VM. And oh by the way, this clustered solution resists software failures in the guest OS or web server by virtue of having two instances.

So you totally misunderstood the software, and try to cop out of it with "we didn't do any optimizations". Sorry - your benchmark isn't just lazy configuration, it's incompetent configuration, and there's quite a bit of difference between the two.

Re:Would be interesting if done right

[merrill77]

We fully understand the different versions available. The version used was clearly documented in the report. Had we not mentioned the verion used, or implied that this was NOT the free version, then we would have "screwed up".

Just because we didn't test the version that you would like us to have tested does not mean that the test is "screwed up". Please feel free to run your own tests on ESX Server if that is what you are interested in.

We ran the tests as designed because
(1) We have 2 VMWare Server (free version) running in our office and were curious about the performance impact under load
(2) We believe that there are a lot of smaller organizations doing the same thing

In the future, I recommend that you ignore evaluations that are not relevant to the needs of your organization.

Re:Would be interesting if done right

[julesh]

Since when are you allowed to post benchmarks of MS software?

Since the enactment of whichever law in your jurisdiction grants you the right to free expression.

Re:Would be interesting if done right

[julesh]

By the way d00d, the VMware license prohibits posting benchmarking results.

Such prohibitions in EULAs are probably totally unenforceable due to free speech issues. IANAL, this is not legal advice, etc, etc, etc.

Re:Would be interesting if done right

[julesh]

I think the parent is objecting more to the conclusion drawn by the summary here ("virtualizing cuts web app performance 43%") which should have been stated something more along the lines of "virtualizing with one of vmware's free products cuts web app performance by 43%". And even that's missing the critical factor that we're talking about serving static files, which is likely to take a larger hit than anything dynamic (which is kind of implied by the rather ambiguous term 'web app').

Maybe a neutral negative mod is needed

[Anonymous Coward]

In the same way as +1 Funny doesn't change your karma, maybe a -1 Retracted or -1 Incorrect that doesn't knacker your karma is needed too...

Re:Maybe a neutral negative mod is needed

[Anonymous Coward]

I'd like the retracted, if only you could mod your own posts down at any time. The -1 incorrect should hit karma.

Virtuozzo Blows

[Anonymous Coward]

Granted, I've stayed as far away from it as I can, but can customers finally use yum without sick workarounds, or is it still effectively restricted to using vzyum on the hardware node itself, rather than the VE?

I'm sorry to say, but XEN kicks Virtuozzo's ass in terms of usability and stability.

VM performance comparison

[chargrilled]

Has anyone done a recent true apples to apples comparison with Vmware, Virtual Server, and maybe Sen on the same exact hardware running the same guest VMs? Obviously it would have to have Windows as the host OS due to Virtual Server being Windows only but I would like to see how the various solutions stack up in different scenarios. Such as IO heavy, CPU heavy, sheer number of VMs that can be hosted, etc.
---
You can use any kind of HTML formatting that Slashdot accepts.
Generated by SlashdotRndSig via GreaseMonkey

really bad report...

[Anonymous Coward]

But it highlights one thing: if you hand virtualization to clueless people, you'll get bad perfs.

It also shows, both in the article and in the comments here, the severe misunderstanding surrounding the concept of "virtualization".

I see lots of clueless people saying "uh, of course, virtualization perfs sucks". I think those people don't realize today's virtualization technology ain't grandpa's past-century emulators.

There are today virtualization technologies that offer basically native speeds. Xen can now run in two modes (para-virt or hardware-virt, the latter if the MOBO/BIOS/CPU supports Intel-VT / AMD-V)... In paravirt mode Xen offers native speeds (the overhead is so small you'll have a hard-time measuring it). Better: network I/O ain't good enough for you? Simply "passthrough" a PCI device (say a PCI network card) to your paravirtualized guest. The guest (and only the guest) is directly accessing the PCI card (no more network I/O problems). But you can't run Windows on Linux using paravirt under Xen...

In hardware-virtualized mode, under Xen (or KVM, which only does hardware-virt), you can run Windows. Network and disk I/O, for hardware virt, at this point sucks. However you can install special drivers in your guest to make it speedier (drivers for Windows under Xen are $$$ and under development for KVM).

But, wait, there's more to come... Next gen IOMMU is around the corner. And as soon as it gets implemented in Xen, the already super-fast virtualized system gets an additional boost and you'll have something even closer to native, even when running Windows under Xen.

If you think "virtualization will always be slower" you need a reality check: the CPU makers are working hard so that the virtualization overhead becomes irrelevant. And suddenly the ones not using virtualization will find themselves with a less capable, less secure, less maintanable box being, in some particular, anecdotical, cases only 0.05% faster.

Virtualization is here to stay and the overhead, already very small today, will keep shrinking.

VMWare EULA?

[jwaters]

Although the focus is on virtualization and its effect on performance, I noticed that the author disclosed that he ran the tests on VMWare. The VMWare EULA states in section 3.3 "You may use the Software to conduct internal performance testing and benchmarking studies, the results of which you (and not unauthorized third parties) may publish or publicly disseminate; provided that VMware has reviewed and approved of the methodology, assumptions and other parameters of the study.".

This relates to the earlier slashdot story where xensource had to redact areas of their performance whitepaper

I wonder if they gave him the green light to report these results on VMWare?

Re:VMWare EULA?

[Quikah]

Yeah, they posted a complete version recently:

http://blogs.xensource.com/rogerk/wp-content/uploa ds/2007/03/hypervisor_performance_comparison_1_0_5 _with_esx-data.pdf

Seriously

[MBHkewl]

This is stupid!

First of all, VMware Server was never intended to be used as a dedicated server, even when put over Linux! That's plain dumb! VMware made it available so that customers can demo the product before buying it.

If anyone wanted to run a dedicated virtual server, then they should use VMware ESX Server.
To obtain semi-realistic statistics on how virtualization is improving, then such statistics should be taken from test on ESX, an environment set up for actual server use; If possible, test the VM on different versions of ESX.

And how is that a load balancing test?? Using one machine only!?? A load balancing occurs when you have multiple machines and distribute the workload on all of them.

Silly test.

Apples to oranges

[billcopc]

Virtualization will slow things down... yes that's obvious, since you're emulating the hardware to a certain extent, and triggering a bazillion context switches every time I/O happens. This stuff is bad for web apps, sure, but normal people don't run just one VM per box. The benefit comes from having multiple VM's running on the same physical machine. For many outfits, it's far cheaper and more efficient to have a whole bunch of little isolated VM's running on one big beefy SMP server, than to have them all as little boxen taking up tons of space and chugging more electricity than Tokyo Square on payday. It also makes it a lot easier to migrate those VM's to bigger/better gear as needed for upgrades or failover protection, often with zero downtime (just a bit of lag during transfer). That's how you can justify spending 25k on a 16-core SMP beast with 32gb of ram and four independent RAID-5 banks. It's still cheaper than 16 separates, not just the upfront cost of equipment but also the ongoing maintenance costs.

Hell I even have a VM server in my apartment, just for the convenience of running 4 distinct OS'es on one neatly stashed box. I don't need a KVM switch or long-ass serial cable, everything's done VNC-style through the VMware console from boot to poweroff. That feature alone makes VM's immensely attractive in many scenarios. For sysadmins, you can do all the work from your desk, no need to go stand in the chilly server room for an hour waiting for a tape restore or OS reload, and you can even script a lot of functionality through the hypervisor API. I'm not saying it's the solution to everyone's problems, but for those things it was designed for, it does them extremely well.

Good News!

[NotFamous]

A study shows that even without optimization, web apps can ran at almost 60% of their native speed using virtualization. Using better optimization, this percentage can be even higher. It's all how you present the data!

Hyperthreading

[Larry_Dillon]

I thought the "Does hyperthreading help in the realy world" issue was related to the way Windows does task switching.

I am under the impression that hyperthreading helps Windows more than Linux because Windows fails to save certain register states, and thus incures a higher cost, in terms of performance, when it task switches.

His test uses Windows VM's on top of Linux. Thus I could see when it could help in his situation, but believe that the generalization about hyperthreading is misleading.

Can anyone clarify this?

Re:Hyperthreading

[insecuritiez]

I've been running a Hyperthreaded Linux box for a few years now. The only test I've done of Hyperthreading is running make -j2 with it on and off. On my P4 2.8 (single proc, 2GiB of ram) Hyperthreading makes a long compilation about 25%-30% faster. I doubt it helps many standard apps much but it really helps make.

Use Solaris Zones or Linux VServer

[Anonymous Coward]

If you are serious about performance of virtualization then you need to look at Solaris Zones, and Linux VServer or Virtuozzo. Running multiple instances of the OS is very inefficient. Zones probably provide the best resource management compared to VServer.

Virtualization comparisons

[dpilot]

This is obviously one data point, and as others have mentioned, not even the best point for deployment.

But there's a whole raft of virtualization solutions available, and that's just in the Linux kernel, not to mention the Windows solutions. It would be fun/interesting to see an updated comparison of the various solutions.

Then for the real benchmark point, it would be good to see what IBM does with the Big Iron virtualization. Intel and AMD are finally adding hardware support, and it sounds like Intel is improving its support on upcoming products, and no doubt AMD will be, as well. But IBM has been virtualizing for decades, and IMHO has probably forgotten more than Intel and AMD have ever learned.

Bogus test -- may not have used the VMWare drivers

[Anonymous Coward]

It says clearly "It should be noted that no tuning was performed on the native server, the virtualized server or the VMs host machine."

Does this mean they did or did not install the guest tools? If they did, well, ignore this.

If they did not it will have a huge impact on the performance of the guest. It's the difference between vmware emulating lots of bits and doodads to make an AMD lance ethernet card and ide interface look exactly like the real hardware so the distributed drivers don't freak out and "here's a packet -- send it."

Nobody in their right mind runs a VM without the drivers, but it is an "optimization"

hmmm.

VMWare, not Virtualization

[bluefoxlucid]

VMWare uses one kind of virtualization, where it intercepts system calls (by loading a module into the kernel that traps faults to privileged instruction execution) and emulates them. When doing heavy amounts of system calls, like using TCP/IP sockets or opening files or reading files that are already open or writing to the UNIX socket for a database (which reads and writes to disk a ton) or whatever, the system slows down massively. Paravirtualizing Hypervisors like Xen rely on the guest OS having its code rewritten to specifically call the hypervisor, in which case the overhead of a syscall is... a syscall. For the most part, performance hit is negligible.

Ancient Hardware

[charnov]

Hyperthreading? Good lord, this is running on ancient hardware. This should be deployed on something that can use either AMD-V or Intel VT. VMWare has hyperthreading support and can show some improvement with "snappiness" but it doesn't seem to help in general throughput over the long run and may contribute to instability.

Usually, the limiter in this type of setup would be IO. When one virtualizes such a setup, you must reconfigure your application to minimize disk IO (with web servers we cache like crazy and really jack up the ram. 16-64 GB is my norm if it is a highly transactional site). 2GB is definitely not enough for IIS & Server 2003 doing anything more than serving static pages.

ASP ASP.NET

[Super Techie]

The headline says the application uses ASP. The article states they used ASP.NET in their tests. This is a large performance difference, primarily because .NET is byte-compiled at execution. ASP is purely interpreted.

Perf

[Anonymous Coward]

I don't see why everyone is saying that's it's a given that running in a VM is slower than running native. I know that makes sense, but will it always be so ?
Look at the Java VM, in some case it can do a better job of running native code, it dynamically knows about using faster hardware and CPU optimizations that the underlying native code did not.
Why cannot a future VM examine common operations and run them faster, like networking code ?

This is comparing single to dual proc

[sheldon]

They state in the test that the servers are dual proc servers.

VMWare Server, the free edition only emulates a single processor environment for your virtualized host.

VMWare ESX or whatever they are calling the expensive thing today, has the ability to give your virtualized host multiple processors.

So it's not surprising that it could only handle half the load, it only had half the processors.

We don't do virtualization for heavy use environments. We do it because different business groups don't want to share servers... that is, they can't agree on maintenance windows, etc.

Re:This is comparing single to dual proc

[RDaneel2]

"VMWare Server, the free edition only emulates a single processor environment for your virtualized host."

Actually, this is not correct. Server does have "experimental" support for supporting a virtual dual-core on a real dual-core... but only on a real CPU with VT (or maybe AMD's Pacifica, but I am not sure about this).

However, the point is still correct - it is costly to virtualize (and possibly even more so with virtualizing multi-core).

no brainer. Don't virtualize stuff with high I/O!

[MrJerryNormandinSir]

Hmmm virualization 101. Don't virulize anything that requires high I/O if you do not have the hardware to do it.
First of all I built an ESX server farm for high I/O apps. Feel free to search for my name at vmware. I used to work
at Welch Foods until the new CIO just mentioned to words "Outsource IT" I left before a decision was made so I can
"stick it to the man". What the heck, the IT Director left anyway. Anyway we created an esx server farm with 8 Dell
6650s, and we had 2 Dell 6680 prototypes to evaluate. Most of these machines had 48GB of RAM and 4 GBIC connections
to our EMC SAN. And... Lots and lots of disk thrown at the databases!!!!!!!!

Another prime candidate to use for a server farm would be a BLADE server with 2 dedicated GBICS per blade.

If you decide to put up esx server on one server with just 2 scsi controllers and expect to run multiple virual machines
to support multiple web applications, then you are a dumb ass and should get out of IT. :)

If your a company that would like an ESX 3.0 server farm designed, then send me an email

My next consulting gig will be Delta Dental.. A very fast esx server farm ro run Oracle on Linux. .. and yeah it can
be done. You need the right hardware and software know how. TUNING.. lots of tuning.

half-assed install = half-assed performance

[James McP]

If I read this right, it's an unoptimized install on a barely server-grade system using a dev package rather than a production package. I would *expect* it to run less than stellarly.

This translates to "brother-in-law IT is crap."

heck, it might be better to say "VMware server will limit performance degradation to 43% when used in a poorly thought out implementation."

CPU's Get Faster

[Nom du Keyboard]

As long as CPU's continue to get faster and cheaper, and caches larger, this is hardly a gloom & doom end of the world. Most systems don't tax their CPU's most of the time anyway, and for the next 3 years at least CPU's will continue to get faster and cheaper.

VMWare w/ Windows VM?

[SponjWorthy]

Um duh! I thought the idea behind virtualization was to run what would have taken two computers on the same computer, and simplify administration. Who really has a mission critical high volume server running in a VM machine? Shakes head... I've used it to take four old (old) servers and put it on one new computer which is actually an upgrade from the P233mhz machines they were running on. I can also see user environments using VMWare to make it easier to restore computers when ID10T's screw something up. But never for a high volume web server. Besides IIS calls way more threads than Apache does to host a page, why would you use IIS for a high volume web server? I do appreciate the study pointing out the obvious!

Re:VMWare w/ Windows VM?

[pandrijeczko]

Um duh! I thought the idea behind virtualization was to run what would have taken two computers on the same computer, and simplify administration.

I don't see how it simplifies administration. Even in virtualization, you have at least two (presumably different) operating systems that need to be managed and update regularly...

The perfect VM ecosystem

[davidwr]

In a perfect world, all VM images would use a common file format and a common subset of emulated devices supplied by all VM engines, e.g. ABC network card, DEF printer, GHI sound card, JKL video card, etc.

Also, the OS vendors like Microsoft and Linux distro vendors should get together with VM vendors and make their installers install the OS "vm-ready." This way, if a customer tried to boot his server's hard drive in a VM environment, it would come up, see the hardware change, and continue along its merry way. Likewise, the OS vendors should create tools to make it easy to move a VM drive to actual hardware. This may mean making an "interrogation CD" that can boot a new box to create a list of hardware, which can then be used to install the correct drivers on an existing VM'd installation in preparation for migration. I realize some of these tools already exist but the industry needs to standardize.

VM engines would distinguish themselves by their relative efficiencies, costs, management tools, additional virtual devices, what hardware devices they pass through to emulated VM devices, what devices were passed through "direct" as virtual rather than emulated devices, etc.

Also, ideally, there would be a popular, easy-to-set up, minimal-overhead, open-source, reasonably-efficient, reasonably-easy-to-manage VM designed for cases where 1 VM ran on each real box. In the simplest case, this would amount to nothing more than

• a minimalist operating system, say, a stripped Linux distro
  
• a minimalist management interface, just enough to create and delete VMs and start and stop VMs, running only 1 VM at a time
  
• the VM engine itself
  
• VM engine is agnostic to guest OS, it doesn't care if guest is Linux, Unix, Windows, or other.
  

Over time, performance is better

[macz]

Because eventually, over time, a disaster will occur causing 100% downtime (or 0% throughput) on a monolithic system. Being able to restart the virtual machine quickly or even transparently on another piece of hardware without doing restores, hardware driver installations, or any number of other time delaying obstacles will increase the overall performance and total throughput of the application. In other words: A monolithic system can calculate pi to 10,000 places faster than a Virtual Machine. But they might both get to 10^10000000000 places in similar timeframes .

Of course it's that way

[ChrisA90278]

Te purpose of VM is to consolidate servers. The reason yu'd want to consolidate is that you find you are running 4 physical boxes and each one is running at about 1/10 of full capacity. So you say "cool I can loose three boxes and cut managmant and electrical power by 3/4. On the otyher hand if you find you need more performance you buy more boxes and "load balance" them

Also everyone knows you loose some performance running VM but if you need to run OS "A" guest or OS "B" host this is the only way. If you need to run Solaris on Solaris there are faster and better ways (zones) and same with Linux (zen)

The whole point of VM in a data center is to put "extra" cycles to better use, you would never use if if "cycles" were in short supply.

Re:Of course it's that way

[jmodule]

I'd mod you up if I had the points. Despite all the typos, you have best expressed statement about why to use VMs in the whole thread.

In other news...

[joto]

Webperformanceinc later released the following reports:

• A faster connection to the Internet may lead to better response time from Internet servers.
• Static pages can lead to better server performance than dynamically generated content
• Powering down servers can save electricity, but leads to up to 100% decreased performance
• Michael Jackson may have taken plastic surgery

Not necessarily true

[Andy Dodd]

Better to say "Don't virtualize stuff with high I/O unless your virtualization solution paravirtualizes the I/O".

As others have posted, VMWare Server sucks for high-I/O situations, while other solutions (Xen using paravirtualization instead of HW virtualization, and to some degree even Xen with HWV) perform MUCH better in high-I/O situations.

At least a few tips regarding disk I/O:
Paravirtualize. Easy with Xen + Linux hosts/guests
When paravirtualizing, don't use file-backed disk images (e.g. /mnt/virtimages/windowsxp.img). Give each "virtual disk" its own partition. Easy with LVM. For example, my laptop's virtualized Windows partition is /dev/system/windows - under QEMU/KVM it's VERY responsive (with the exception of guest video resolution changes - QEMU+KQEMU handles this well but QEMU+KVM does not. The result is a blank white window with maximum guest OS CPU usage for a few seconds.)

Basic computing

[bean123456789]

This is pretty basic stuff, I think everyone knows that when you use a VM it won't run as fast as hardware. This is a classic tradeoff of management vs. speed. What is more important? Choose your solution from there. Although I think a good way to setup your servers is to use a hardware server and have a VM box for cutting over in case of failure on the main server. This way you at least have something running and aren't SOL if your server dies.

Re:Fast Virtualization: Xen, KVM, Virtuozzo, GSX,

[ErMaC]

I would beg to differ.
We've done testing with many tools, and VMware ESX is the fastest true virtualization suite that we've tested.
First off, Virtuozzo isn't a real Virtual Machine hypervisor at all, it's a way to jail applications in a Windows environment so they don't interfere with each other; it doesn't create all-out VMs like the others.
Second, your view on "hardware virtualization" assist is flawed - Intel VT and AMD-V (which are the two virtual assist features out there) both simply make it *easier* to write a Virtual Machine Monitor, not faster. VMware doesn't require AMD-V or VT except in a specific case (64-bit VMs on Intel hardware) because VMware's Binary Translation stuff is *faster* than the built-in CPU instructions. They've had more than half a decade to perfect their virtual machine monitor, and their performance is very, very good.

You see all these new Virtualization products out on the market now that require VT or AMD-V (Xen needs it for windows VMs, KVM needs it period) because now, anyone can write a reasonably-performing VM Monitor by using the build-in CPU features. But that doesn't make it faster!
See this presentation from VMworld 2006 for more info:
http://download3.vmware.com/vmworld/2006/tac9463.p df

It is relevant, we just had a similar issue

[hooykaas]

The company I was a consultant for recently, had a policy using VMware for almost all windows servers. We were installing a new type of java middleware software, and noticed a significant performance overhead from using VMWare. We estimated the overhead might be as high as 50% (that 43% sounds familiar), while VMWare usually claims only a few percent overhead, for most applications.

The important part was that the middleware product was expensive and the license type was a per processor license. So a 50% overhead, implied that the number of processors needed could double, and thus the license costs could double, which would a be very significant cost increase.

So it is nice to hear similar experiences form other applications. My own view is that VMware might be very well for many situations, but there are exceptions. I don't have any proof, just a gut feeling, but is seems that having java applications with high numbers of threads can suffer a significant performance hit.

Hmm

[Anonymous Coward]

It depends on what type of virtualization you do and how you do the virtualization and stuff.

They should have ran Linux, Apache and PHP virtualized on a Linux machine, instead of Windows, IIS and ASP on a Linux machine.

But the conclusions don't match the research

[g2devi]

Th key thing to not is that their tests don't substantiate their conclusion:
> These results indicate that a virtualized server running a typical web application may experience
> a 43% loss of total capacity when compared to a native server running on equivalent hardware.

This may lead to people believing that virtualiz]ation just isn't worth the advantages. The key problem is that there are several virtualization schemes. Off the top of my head, I can list:
        * Xen
        * KVM
        * Linux-VServer
        * OpenVZ
        * User Mode Linux
        * lguest
Each has it's pros and cons in terms of overhead, maintenance, and performance. Generalizing based on one VMWare configuration, is just plain foolish.

Let's not forget the old chroot approach that gives you 90% of the advantages of virtualizing web servers with few of the disadvantages.

The key thing to do is to pick the right technique for the right task.

Slashvertisment

[teneighty]

The test was so poorly performed that the article was useless. It is little more than a poorly disguised ad for their load testing product. All I want to know is: how much would it cost for me to post a similar slashvertisment as a'story'?

KO (coke) used VMs for threading problem

[scarbelly]

In the late 90's I heard about CocaCola in Atlanta using virtual machines to increase the number of users per Lotus Notes server. When running Lotus Notes servers on Windows they found that they ran out of threads even though the procesor and disk load were not maxed. By running linux on the HW and running multiple Windows VM's for Lotus Notes server they were able to have more users per physical box.

At the time they were running out of server space and power in the server rooms so there was a rule that one server had to be sunset before another was added. By adding multiple virutal machines per server they were able to increase the number of users w/o adding more physical servers.

SQL Server was virtualized too

[sr23]

I have exchanged a couple of emails with Chris Merrill (the person who did the test). He confirmed that SQL Server was used as the backend for the ASP.NET Issue Tracker System (the app they were testing) and that SQL Server was virtualized too (I am not sure which version and which edition of SQL Server they were using). My experience have been that database servers do not virtualize well, at least not the ones under any kind of decent load. Unless we are talking really small or very mildly used web apps, you do not normally put web tier and database tier on the same box (at least not in the Microsoft world). It would be very interesting to see the results with SQL Server removed from the equation (running on another server).

VMWare Server instead of ESX?

[subsoniq]

why in the bloody hell were they testing with VMWare Server 1? If they want to test a production business app then they should have used a tuned VMWare ESX machine and tuned guest OS. As far as I'm concerned VMWare Server, while a good product (and better than MS's server product) is not something to be used in production, it's more for labs/dev environments, or very small companies that have no budget for virtualization and can live with the performance decrease.

I also see they used cheap hardware, something meant for small SoHo or branch offices. Should have tested on a Poweredge 2950 at least, with good storage.

vserver, no performance drop

[a1mint]

xen, vmware, why so difficult? Why not run vserver (or similar) in a unionfs layer? You can still have all the benefits of migratibility without the performance hit.

5% Virtualization-Overhead on Mainframe

[Anonymous Coward]

Microsoft itself measured 5% virtualization overhead on Mainframes. http://download.microsoft.com/download/7/3/e/73e77 129-db34-4c95-b182-ab0b9bd50081/MainframeBenchmark Proj.pdf

And they are not talking about a single image, they are talking about 60 or 90 images in this 2003 Mainframe-Benchmark.
Would be interesting to see what is the performance and throughput penalty with a 90 image IIS-Setup.

But what vendors are cooperative?

[tepples]

Why would a judge be needed for Microsoft to modify their own operating system for paravirtualization? Because Microsoft is a convicted monopolist, and remaining uncooperative with paravirtualizer vendors helps Microsoft maintain its monopoly position.

What is needed is either access to source code (and legal rights to modify it), or a cooperative vendor. In particular, free software, is certainly not needed, nor open source. A cooperative vendor, and/or source released under an NDA is perfectly acceptable. In theory, what you say is true. In practice, the vendors of the three leading proprietary operating systems for x86 servers are uncooperative.

Hyperthreading

[Taliesan999]

They key thing I got from this was that VMWare Server not ESX doesn't seem to allow the virtual machine to take advantage of hyperthreading. With hyperthreading disabled, the performance of the virtual machine was MUCH closer to the performance of the real machine.

Having said that, as other posters have mentioned, this is with VMWare Server, not ESX which is the VMWare product meant for high load environments and not labs.

Re:no brainer. Don't virtualize stuff with high I/

[Anonymous Coward]

Having read your comment, reeking with dumb-ass attitude, and splashing internal information about your previous and next employer's IT departments all over the place, I'd just say don't hold your breath waiting for that phone call for your next gig...

Re:SC1420 CPU type a mystery

[Douglas Goodall]

I have spent a significant amount of time and trouble trying to find out exactly what processor is used in the SC1420. I ran the Intel CPUID program and tried to feed the results into the Intel processor specification decoder. In the end, I was unable to match the processor with any certainty against any specific CPU. I contacted Intel and provided hem with the CPUID results, and they refused to answer my question and said that it was up to Dell whether or not they wanted to disclose the processor they put in the system. I asked Dell what the sPsec number was on the processors they use and they rerfused to answer my question. The only way to find out for sure is to pull the CPU out of the box and scrape off the thermal goo and read the sSpec number off the chip. Of course doing so voids your Dell warrantee and leaves you with a dissasembled computer. Becase of this stupid situation where I am not allowwed to know what part I have purchased, I will no longer buy anything from Dell.

Virtualization Performance Testing

[Anonymous Coward]

Interesting paper, but not very applicable to an Enterprise Data Center. No one in their right mind would even think of putting VMware Server 1 in production: test and development, maybe, but production - never. Also, no one in IT management or administration would ever think of putting a single VM on a Virtual Server. The minimum would be two or three.

Also, the article did not state the specifications of the native Windows 2003 server. Did the native server have more or less CPUs? Did it have the equivalent amount of RAM? Were the NIC cards the same or at least the same speed? In performing the tests, the host system should have one CPU reserved for itself and about 512 MBs of RAM. It should also have a dedicated NIC card. So, if the native Windows server had two CPUs, 2 GBs of RAM and 1 NIC, the host server would need 3 CPUs, 2.5 GBs of RAM and 2 NIC cards. At least with this configuration we are getting closer to comparing apples to apples.

I would expect a performance loss of some percentage using VMware Server, but not 43%. The author did say that no tuning was attempted and I would expect that with proper tuning and the above hardware specifications it would be possible to achieve an acceptable 20-30% performance loss. Even with ESX there will be approximately 5% loss due to the overhead of the kernel, but this is very acceptable.

The idea behind virtualization is not one-for-one performance matching, but to consolidate multiple physical servers into one. If you have 5 servers running with a peak 10% CPU load and equivalent memory and I/O loads, then they are candidates for virtualization. IT managers are looking for lower Data Center operating costs more than they are looking for higher performance models. If it means they need to add an additional VM to cover the performance losses, that's acceptable as long as they get the power, cooling, floor space and general operating costs reduced.

In closing let me restate that the fundamental issue with doing a one-for-one performance analysis regarding virtualization is flawed. No one expects an application running in a VM to perform the same as it would on a stand-alone system.

VMware response

[Anonymous Coward]

VMware has posted a response: http://blogs.vmware.com/vmtn/2007/04/response_to_l oa.html

# The test results are another example of "apples to oranges" comparison, and the reason VMware requires a benchmark review to ensure that benchmark test methodology is correct. Here are more details about how the physical and virtual configurations differ:

      1. CPU: In this case, the physical environment consists of dual Intel Xeon processors with hyperthreading enabled i.e. there are 4 logical CPUs in the physical environment. The virtual environment details are not provided, but assuming default values, we imagine the virtual machine is using a single virtual CPU. So in essence the test is comparing results from a 4-processor physical environment to a 1-processor virtual configuration. This can have a huge impact on multi-threaded apps such as this .Net application.
      2. Memory: The physical environment used 2GB memory available to the machine. In the virtual environment, the VM was also assigned 2GB (the article implies that the physical machine has more memory). While all the details are not available, this memory configuration may result in swapping since the host operating system and VMware Server have their own memory requirements.