Desperately Seeking Xen

AlexGr sends us to an excellent article on the state of Xen by Jeff Gould (Peerstone Research). He concludes that the virtualization technology has some maturing to do and will face increasing competition for the privilege of taking on VMWare. Quoting: "What's going on with Xen, the open source hypervisor that was supposed to give VMware a run for its money? I can't remember how many IT trade press articles, blog posts and vendor white papers I've read about Xen in the last few years... The vast majority of those articles — including a few I've written myself — take it as an article of faith that Xen's paravirtualizing technical approach and open source business model are inherently superior to the closed source alternatives from VMware or Microsoft."

Comments

Re:Need a special processor

[Mostly a lurker]

It is true that Xen requires special hardware to legally run MS Windows. It is also better for performance, generally, to have such hardware. However, there is nothing stopping you from running Xen on pretty much any computer you are likely to own as long as the VMs are Linux based.

VirtualBox performance

[mwilliamson]

It seems that VirtualBox.org's product, fully virtualizing a copy of XP on my non-VT machine under a linux host OS, totally runs circles around Xen even on VT hardware as far as performance is concerned. Integration into the host enviroment is also quite beautiful. Why is there seldom a mention of VirtualBox in this arena?

Because....

[Ayanami Rei]

virtual box is basically QEMU with a much better KQEMU component that they developed on their own. This isn't very interesting because this is the same thing as VMWare or any other closed source Ring0-in-Ring1 emulation using polymorphic code.

Host OS the one with better drivers

[interiot]

If I have to maintain two separate OS's, I'd rather have the outermost OS (host OS) be the one that has the best drivers, the most hardware support. Also, since very few virtualization solutions work with 3D gaming (and even the one that does, it still has large overheads I think), you want your host OS to be the one that has all the games. So, for my purposes anyway, I need Windows as the host OS, and Linux as the guest OS. Xen doesn't run under Windows, only Linux. So that leaves me with either commercial virtualization software, or a few open source projects that haven't matured yet (eg. coLinux).

(granted, having Windows on the outside makes your machine much less secure than the other way around, but personally, I'm more interested in having all my peripherals work the day they're released, and having all my games available)

Re:vm ware

[Per Wigren]

If you want to run multiple linux instances on the cheap then xen is the way to go at the moment. Except that OpenVZ is a better way to go in that case. If you are only going to run multiple instances of Linux, with OpenVZ you don't need to preallocate a fixed amount of memory for each VM, the root filesystem can be a subdirectory of the root OS instance's filesystem, among many other things. It can do just about everything that XEN can do, including live migration to other physical nodes.

Re:what went wrong is

[stevey]

Not true.

If you have VT-capable hardware then you can run Windows under Xen. You do need the hardware to support it though, and that is a problem for some home users. Recent AMD and Intel chips have slightly differing VT support but both work.

I run Xen at home along with xen-tools (which I wrote) to easily create new Debian guests on demand. These are used for software testing, hacking, and general service isolation.

I think Xen is just now reaching "mainstream" in the sense that you don't have to be an early adoptor or major tinkerer to get it working. Now that distributions are including Xen kernels in their newer releases it really us available for all.

Using it...

[dmayle]

You can choose to believe the hype or not, as you wish, but I'm using Xen in my production environment, and it's simply fantastic. I've got friends with companies who are doing it as well, and it really changes how you think about administration.

Of course, there are some learning curves. For example, how you manage 3-7 servers is completely different from how you manage 20-30, even if they are all virtual. There's a lot more emphasis on system images, isolating functionality, reproducing configurations. On the other hand, dev environments are so much easier to build-up and tear down.

I just wish the OpenBSD port was in a usable state. The mercurial servers hosting it are often down, and even when they're up, I haven't been able to get a working kernel compiled from the sources (even after doing some of my own bugfixes). And last I saw on the Xen lists, Christoph Egger (the guy doing the OpenBSD port) submitted a security patch related to stack slamming, and the Xen guys were kind of like, "meh, security's not really a priority..."... Oh well, here's to keeping my fingers crossed

The Problem With Xen

[Aeonite]

Is that Gordon Freeman ruined it.

Xen "Just Works" (I know. I use it every day)

[eno2001]

Not only that, but I've been running it in a production environment for about a year and I'm about to deploy a HUGE set of servers as VMs using it. Xen beats VMware in one arena: price. If you use the open source version (which I'm doing) it's free. Only VMWare's ESX can compare to Xen. And unlike some people here have been saying, you DON'T need a special processor for Xen unless you plan to virtualize Windows. In my environment, I'm only virtualizing Linux, so I can use regular x86 CPUs dating back to 1998 for Xen. The only exception is the deployment of Zimbra I'm going to do. It requires Redhat Enterprise Linux 4 and NPTL, so I can't run it paravirtualized, it must run HVM which requires the special processors. However, who today isn't getting new hardware with HVM support?

Currently my two Xen servers here at work serve out about four VMs (all paravirtualized on older hardware) for critical and I/O intensive tasks like proxy servers for nearly 1000 machines, or the firewall syslog server for a dual T3 link with about 5000 users behind it sucking the bandwidth dry. So you can't claim it doesn't perform either. Now, if you want point and click administration and an easy set up, then yeah, Xen is behind the times. But performance wise it's leaps and bounds above VMWare. Trust me, I was a VMWare fan before you were in virtualization diapers. And I still am for some applications. But for places where I need something to be cost effective AND give me the features of VMWare ESX, Xen is the ONLY answer.

Lies, all lies

[C_Kode]

Another question hanging over Xen performance concerns the availability of paravirtualized drivers for Windows.

This isn't true completely. The problem is you cannot get these drivers by downloading the OpenSource Xen. You MUST buy the XenSource version. If you run Windows on the *complete* open source version, your network throughput is going to suck like you would not believe. You have to use the XenSource version to get the paravirtualized drivers that bring the network performance closer to what it should be. Virtual Iron has a set of drivers also. (which I believe are better than Xen's, but don't hold me to that)

I found a lot of great insight about virtualizing from Xen to VMWare to Virtual Iron and others on this site. http://ian.blenke.com/xen

Re:vm ware

[gtrubetskoy]

Except that OpenVZ is a better way to go in that case. For fairness it should be mentioned that aside from OpenVZ there is also Linux VServer which does a few things better than OpenVZ (though OpenVZ does some things it does not). Our preference has always been VServer, it's a well-run project with emphasis on quality and well thought through design rather than quantity.

Xen (and virtualization) is for the Enterprise

[xzvf]

While it might be nice if all these things are easy and work well for the hobby crowd, the real money in virtualization is in the enterprise space. Most servers in enterprise environments run 15% max and are refreshed every 3-5 years. The special processor matters less in that case, and the competition is between a mature VMWareESX server (not free), a hardware based IBM and Xen. Microsoft is a surprisingly minior player. VMWareESX server is very good for x86 consolidation and saves customers money, but is very expensive. It is still the best option for Intel based consolidation. Xen has deep penatration in enterprise lab environments. It is just getting the enterprise management tools to move into real production. IBM is very good at virtualization and stability, but on proprietary power and mainframe hardware. Xen will be fine, because the market is very immature, but expect more seamless and non-attrusive virtualization on the desktop.

Re:Xen (and virtualization) is for the Enterprise

[The Second Horseman]

Expensive? Not really, if you compare the costs of actually getting that number of servers. Given the feature set you get, it's pretty modest. If you work in an educational setting, it's even cheaper. You can get VI3 Enterprise and a tier one server - 2U rackmount, dual quad-core system (2.4 ghz) with 16GB of RAM, dual power supply, 6 hour CTR service, an additional NIC and a 4GB fibre channel card and about 512GB of local storage for about $16,000. Depending on the size of the VMs you need to run, you can easily get anywhere from 12 to 20 VMs on that server.

We're entering our fourth year of server virtualization under VMware, and we would've sunk without it - trying to meet customer demands in our space and budget would have been impossible.

Factoring in things like the cost of gigabit ports on a data center switch, cost of power outlets (distribution is often as much of a problem as capacity), and the cost of fibre channel switch ports, there's a huge savings per server. We've got 42 VMs running on three servers similar to the above (one 2xdual core, two 2xquad) and an older server still running VMware ESX 2.5.x slated for replacement next year.

And yes, fibre channel. It's still cheaper than 10GB ethernet, and can run at 4GB.

Re:what went wrong is

[Anonymous Coward]

Yes, but as he already told you, what you know is out of date. You can run an un-modified Windows on top of Xen provided your hardware supports the Intel or AMD VT extensions.

Guaranteed Results

[catdevnull]

If you want to get a colorful thread of comments started on slashdot, there are 3 ways to do it with guaranteed results:

1) Say something bad about linux (or about Apple).

2) Say something good about Microsoft (or about Apple).

3) Throw a grenade in the room about Open Source software like this:

The vast majority of those articles -- including a few I've written myself -- take it as an article of faith that Xen's paravirtualizing technical approach and open source business model are inherently superior to the closed source alternatives from VMware or Microsoft.

I'm not making any value judgements here--I'm just amused.

Re:Xen "Just Works" (I know. I use it every day)

[div_2n]

The only exception is the deployment of Zimbra I'm going to do. It requires Redhat Enterprise Linux 4 and NPTL

Last I checked, Zimbra runs on Ubuntu 6 just fine.

Re:Xen "Just Works" (I know. I use it every day)

[eno2001]

It depends on what I'm doing. If you weren't trying to be cute, I'd say you were trolling. In reality, it's very common practice to use LVM to clone a filesystem, make some changes to the various files that set IP and hostname as well as other unique host settings and bring up alternate "Test" VMs on a Xen box. So some days I might be running three VMs other days eight or ten. It all depends on what I need to do.

As an aside, I forgot to mention that there are NO other products other than VMWare ESX that offer "live migration" of a running VM from one hardware host to another. That's right... you can take a VM that is running with many users actively using it and move it from one physical box to another with only a few milliseconds down time. The users NEVER notice. The free VMWare server can't do that. Micrsoft's Virtual Server can't do that until they have a hypervisor. And there really isn't anything else that can.

Xen's Maturity

[Krondor]

He concludes that the virtualization technology has some maturing to do...

I RTFA and it says very little about the maturity of the actual Xen technology. The article is more a point about several non-related factors;

1.) There is a lack of pretty management interfaces.

True, but these are in the works from Red Hat, Novell, XenSource, and various other ends. Already some of them look pretty promising, but if you are a real admin you don't need them in the first place. There is nothing wrong with using the command line tools to manage your Xen virtual guest environment.

2.) There is a lack of references for companies using Xen.

How does this relate to the viability of the Xen virtualization? Yeah it makes management feel nice and fuzzy that others are using something, but this does not relate to how well the Xen technology performs. I also suspect that like many open source projects, there are many people using it that do not report it. Novell has personally contacted me and my company to ask us to assist in their new paravirtualized Windows drivers initiative and then be a reference for the technology. It seems that at least some companies are moving to address this, at any rate.

3.) There aren't many benchmarks about Xen versus VMWare.

VMWare does not allow benchmarks they do not approve of. It's in that draconian EULA you agreed to by using it.

4.) It's awkward to paravirtualize Windows.

Yes, it is. Novell signed the soul sapping agreement with MS and as such is pushing some paravirtualized drivers for Windows. The article continually talks about woes with Xen on Red Hat. Red Hat didn't sign the agreement and will require some much more intelligent coding to make this happen. It might never happen, so for Windows it's full virtualization with VT (or AMD's equivalent) or bust. Sorry, use SUSE for it or use full virtualization. It's an MS issue not a Xen issue.

5.) MS's new Viridan Virtualization Platform is using paravirtualization as well.

Yep, that should be a testament to the approach versus VMWare. Though it is interesting that VMWare now has a Linux kernel virtualization implementation similar to KVM. It seems VMWare is headed to paravirtualization as well. Obviously Xen did something right.

6.) There is a lot of competition.

True. How again is this relating to Xen as a virtualization technology.

Again, I'm not saying Xen is perfect. It definitely has issues and room to grow. I'm just saying that the article makes little, if any, relevant points to Xen's virtualization technology.

Re:Xen's Maturity

[FireFury03]

For example, Xen is a bitch to setup with any sort of non-trivial networking environment (eg: multiple vlans, bonded interfaces, etc).

I'll agree with this, although it isn't the hypervisor's fault - it's the userland stuff that's at fault. For example, Xen doesn't appear to support IPv6 *at all* in routed mode, I had to hack up my own scripts to do it (and I'm seriously considering moving over to bridged mode in an effort to simplify and standardise my system). But I'm curious - do other virtualisation systems handle these sorts of things any better?

I've never understood why some people - OSS unix geeks in particular - consider the quality of software to be inversely related to how easy it is to use.

I certainly don't. However, experience has taught be that software with shiny GUIs is often inferior in other, more important areas. That's not to say that I think making things hard is a good thing, but maybe the programmers are fixing problems they consider to be more important. Afterall, Free software is usually written out of necessity (i.e. developers implement the features that _they_ want) rather than having some corporate agenda to make it "easy" for "normal people".

Also, in my experience software with GUIs is also often missing a decent commandline interface - GUIs are all very well but once you have learnt to use a commandline interface then it is (a) much faster to use and (b) usable over a low bandwidth network connection (I don't want to try tunnelling an X11 based configurator over GPRS just to tweak a setting on my server).

If I'd known before I started pursuing Xen what I know now, my advice to myself would be the same to my advice to most people looking into enterprise-level virtualisation - get VMWare ESX.

I've not had any significant problems with Xen (other than the above note about IPv6 support) so in my case VMWare would be a waste of money. I guess each person's needs are different, but for me the act of actually going and buying VMWare (and having to keep it up to date) would make it more effort than just using Xen which happens to be already there on recent distros and Just Works out of the box for most situations.

You might think it's expensive, but it's almost a certainty you'll spend more money on Xen in wasted manpower trying to get it up to half the functionality and relability.

I'm not sure where your reliability concerns come from - I've been running a number of Xen VMs across a number of real machines for quite some time and I've never had one break on me. Also, as mentioned, for what I'm using virtualisation for, I would be expending more man-hours administering VMware than I am administering Xen so I think your arguement is possibly only applicable in some very specific situations.

Re:Xen "Just Works" (I know. I use it every day)

[Courageous]

The free VMWare server can't do that. Micrsoft's Virtual Server can't do that until they have a hypervisor. And there really isn't anything else that can.

Well you can try to do that with Xen if you want, but you might be sorry. :) But I suspect you knew that.

Hopefully the Summer release remedies this situation.

C//

Xen Management Apps Are Not Good.

[tji]

I have been trying to use Xen at home to test it out and compare it to VMWare, which I've used at work. Once you manage to get Xen clients working, it's fine. It does a good job of running VMs, and can be used to partition resources on a powerful machine.

But, the main problem is the steep learning curve for getting Xen running in the first place. The (python based) management GUIs included with Fedora or Ubuntu are weak at best (although, slowly improving.. the UI in Fedora 7 does manage to make setup easier than the command-line alternative). The ongoing management / monitoring of VMs is okay, but weak in comparison to VMWare.

There are also a lot of little quirks in Xen. Installing Win2k in a client VM required a lot of searching for how to attach an ISO image to a running VM (it's not a simple GUI operation like in VMWare/Parallels/VirtualPC, it requires a terminal command with unintuitive options, which never worked for me.. I finally dug out my CD and got the physical CD drive to attach to the VM). Windows VMs have an odd issue where the mouse pointer is offset form the actual pointer (it's a known issue, and is helped by turning off mouse acceleration in Windows preferences, but it is still a problem). Installing client VMs can be challenging.. Ubuntu feisty wouldn't install until I set the VM as a Solaris client, and after a few other tweaks it finally installed and worked fairly well.

Most of the Xen problems are solvable, after playing with command-line tools, figuring out poorly documented parameters, and lots of googling. At the end of the day, it's one of those "Xen is free, if your time has no value" type things. VMWare Server is probably a better option if you just want it to work for home/free uses. For commercial use, VMWare ESX Server is the way to go. It has simple VM setup for many client OS's, excellent management of large groups os Hypervisors and virtual machines.

The commercial alternative from XenSource (free to use, but limited to 4 VMs; or less restricted versions for increasing $$) offer a better management UI, but are too restricted for my taste. The management app is much better, but not as good as VMWare.. If I'm going to pay for one, I'll go for the best option.

Data Center USA

[fyoder]

I stopped reading the article with this quote:

Oh my. Editable XML configuration files, obscure command line interfaces, grayed out options in the GUI? Thanks, but no thanks. This thing doesn't sound like it's ready for prime time in Data Center USA.

Are sysadmins at "Data Center USA" morons? "Oh nooo, command line time, I hate that. Oh nooo, my option I want is all grayed out! Help me, help me! Oh I am so sad now."

Deploying vm stuff is not the same as using a word processor. "Data Center USA" is in real trouble if their sysadmins aren't any smarter than regular desktop users.

Re:Data Center USA

[turbidostato]

"Such tools that people want to rely on, oh I don't know, all the time, require good management tools in order to set things up in a straightforward way that can be documented and can be reproduced"

Like, uh... a script? I had always problems trying to understand the rationale of "documented an reproductible" and "GUI" in the same sentence. Can you really talk about "can be documented and can be reproduced" on bold face when all you have is a doc document an some screen captures? Can you really talk about "can be documented and can be reproduced" when something suddenly starts to fail and you don't have the slightest idea about what was changed from yesterday?

"Do you really think editing a bloody XML file is a good idea?"

Not at all. But the ability to diff it to yesterday's version to see what changed it is. As it is running an script from cron instead of waiting for somebody to click on a GUI's box. As it is to have an scripted deployment environment you can guarantee repeatability with instead of depending on the read abilities of a sysop going through a doc with screen captures.

Re:Timely for me!

[TheRaven64]

There was some interesting work presented at the XenSummit which should be making it into the main tree eventually on 3D support in Windows. The idea was that the memory layout was adjusted slightly so that Xen and dom0 lived nearer the top, and Windows in an HVM domain lived at the bottom. This allowed Windows to use existing 3D drivers, without having to do any address translation when performing DMAs (not an issue if you have an IOMMU, but hardly anyone does yet).

The problem with giving access to hardware to guests at the moment is that without an IOMMU, any DMA request the driver issues will read or write memory from a physical address indicated by the driver. In a virtual machine, what the driver thinks is a physical address is actually a virtual address. This means a DMA request will read from or write to an arbitrary memory location. By putting the HVM guest at the start of memory, this translation is the identity function, so the driver will work. The only downside is that you lose protection from other domains; a malicious driver can still damage your other VMs or even the hypervisor.

Two words: OpenVZ

[pyite69]

If you are running the same OS on each VM on a server, OpenVZ is the best.

Performance is great, good control over resources (with the glaring exception of disk IO operations, which they are working on).

Re:Timely for me!

[MajinBlayze]

The problem with Hardware acceleration in VMS is fairly straightforward. The driver sends information such as 'use bitmap located at pos x in memory' The way memory mapping works, the VM might be given a chunk of memory (i.e. positions 100 to 200) and sees this as 000 to 100. for the VM, x = 010. When the card tries to access that memory, it's memory that might be assigned to a different VM, and thus garbage. Unfortunately, this generally requires the cooperation of the drivers.

fortunately, VirtualGL is working on a generic fix, although it's still immature.

Xen rocks: In production

[Tracy Reed]

Xen saved my former employer a bunch of money and gained then great flexibility and reliability. They use an AoE (ATA over ethernet) SAN so the compute nodes are totally diskless and all of the data and root filesystems are on the SAN. Now they have email, database, web serving, nearly all of their critical functions in a highly available xen-aoe cluster. I am working with them to release all of the codes and configs in production and we are setting up a website at xenaoe.org (not up yet, but soon) to host the project.

Here is something I wrote up about this architecture for the company when the project went live:

What is Xen?

Xen is a free virtualization system similar to VMware but different. It allows us to run multiple servers/operating systems all on one physical piece of hardware while providing isolation between them.

What is AoE?

AoE is a SAN technology. Similar to Fibrechannel (but far less expensive) or iSCSI (but far simpler and more efficient).

What are the advantages of Xen and AoE for our company?

Xen allows us to more efficiently utilize our hardware resources. The majority of cpu power on your average computer goes unused. Even on servers. They just sit there waiting for something to happen. Even if we get a web request every second the time between one request and the next is an eternity for a cpu running at 2 gigahertz. But powerful cpu's are needed for those short bursts of activity. By using Xen to run multiple servers in their own domains (areas of memory) completely isolated from each other on the same physical hardware we can squeeze more utilization out of our existing CPU's/servers. This means we can get by with fewer CPU's, less rackspace, use less power, and require less air conditioning. By encapsulating the servers into this sort of infrastructure it also allows enhanced management capabilities by allowing the administrator (such as myself) to be able to get console access on the server or restart the server while remote instead of having to drive to the datacenter (which in our case is a 30 minute drive down to Kearny Mesa).

AoE allows us to put a bunch of disk in relatively inexpensive and low CPU powered servers on the network and allow the rest of the servers to access it exactly as if the disk were locally installed in that server. This is advantageous because we can now aggregate all of our disk into one system and treat it like a pool of storage where we can dole out an appropriate amount of disk to each server (often only 10 or 20G is needed) instead of having to put in a dedicated 250G disk which is the minimum you can easily buy these days and waste a lot of disk and power to run it.

The combination of Xen and AoE allows us all of the above plus some interesting fault tolerance abilities. There are now two levels of redundancy in our disk systems and an extra level of redundancy in the cpu's also in that if one cpu fails (or the associated motherboard, RAM, or network card) we can easily switch the servers that were hosted on that machine over to another cpu on the network with either zero or very minimal downtime whereas previously that kind of failure would have required me to drive down to the datacenter and shuffle hardware around or buy new hardware to replace the failed system which all takes time and can result in prolonged downtime.

Re:6ms???

[MajinBlayze]

YMMV, depending on usage during the time of the switch, but Xen starts migrations by copying over memory *while the original VM is running*. Then, the original VM is suspended, checked one last time for data consistancy (the delay), then the VM is brought back up by the new host.

[PDF warning] Live Migration of Virtual Machines

Re:Currently using Xen...

[BJZQ8]

Well, I've done numerous tests of live migration, and it works for me. Do I know something they don't? I don't think you can say either way. Perhaps on somebody else's network, it might not work. But I've done migration under heavy load and not had any failures, zombies, or crashing of Dom0. I had to clarify the 6ms delay in another post. Here's the way I put it, simplified because I screwed up the formatting...

ping...64 bytes from xxxx...5 ms
ping...64 bytes from xxxx...5 ms
ping...64 bytes from xxxx...11 ms
ping...64 bytes from xxxx...5 ms
ping...64 bytes from xxxx...5 ms

    The bump to 11ms was the transition between real machines.

  We use it because we like to have lots of machines doing different things at different times. The developers have wild ideas, so we give them what they want as far as an OS, and create and destroy as necessary. We keep the migration as an option in case of failure; we don't do it on a regular basis.

Re:Xen "Just Works" (I know. I use it every day)

[mtvsucks]

One of the frustrating things about all of the virtualization stuff is how quickly the products are moving and what feature sets are gained and lost but what products. What was true a few months ago about live migration isn't true anymore.

ESX can do live migration from one physical server to another provided the virtual machine image lives on a shared storage.
Xen can do live migration on a paravirtualized image from one physical server to another provided the image lives on shared storage.
Xen cannot do live migration on a fully virtualized server from one machine to another.
KVM doesn't specify full or para virtualization on their migration page, but it does some sort of live migration, and I'll bet it needs shared storage.

Re:Currently using Xen...

[Courageous]

I understand the latency of the switchover. It will be dependent on the size of the volatile set of memory that needs to be transferred between the save/restore cycles. I.e., this will be virtual machine-dependent, and tend to increase linearly with the virtual machine's memory footprint and memory utilization.

"Data center readiness," to me, does not mean a few servers running Xen. It means many, many servers, taken from at least superset of servers taken from all the mainstream enterprise server vendors, in all the common configurations. IOW, if I have a set of Dell 1955 Blades with Net App 3050 NFS storage heads running Xen and it crashes during live migration, (and I do), Xen is not ready for data center deployment, QED.

An example of someone else who's jimmied together a successful rig does not serve as evidence of "data center readiness." I have tried SLES10 out of the box, SLES10+patches, RHEL5, RHEL5+patches, and Open Source Xen with careful compilation of specific kernel options... all with live migration. They are all terribly unstable on our hardware configuration.

Show me an enterprise deployment of hundreds of Xen hypervisors deployed servicing guests with a minimum of four nines of availability guarantee with a contractual financial penalty for availability violations... then we can start talking seriously about the "data center readiness" of Xen.

Today Xen is /not/ "data center ready".

Maybe next year.

Not today.

If you ask me, Xen's going to get clobbered by KVM; a different subject for a different day.

C//

Re:Xen kernels are nearly useless

[drsmithy]

Indeed, why would they?

Because being able to dedicate a "machine" to each service rather than trying to run dozens of different services on the same machine vastly simplifies operations.

Pay attention now. This is not how Google runs their datacenter.

How Google runs their datacentre is not relevant to most people, who have vastly different requirements, budgets and capabilities.

Shop around. One can find power-efficient 1U boxes. Sometimes non-rackmount is better, including weird stuff like the Mac Mini. Be willing to look beyond Intel and AMD. VIA makes some low-power chips.

Indeed. Then instead of the 2-4 power connections, ethernet connections, fibre channel connections, the cooling capabilities, electrical capacity, 4RU (or less, with other hardware options) of rack space, etc you need for a single machine running 30 VMs, you need *60* power/ethernet/FC ports, higher cooling and power needs, 30 rack units, 30 KVM/serial/RJ45 ports, etc.

Price those out in a datacentre and suddenly that "expensive, single point of failure" becomes cheap and easy to turn into 3 or 4 machines running VMs.

That's just the basic physical footprints - this is before even getting into the _manageability_ advantages of VMs over physical machines (eg: being able to roll out new servers by running an install script and coming back 20 minutes later, instead of having to get physical hardware specced, ordered and installed).

Re:Fundamental performance issues

[Courageous]

20 Dell 1955 Blades; 16G ram; 70GB SAS 10K drive (one) on which ESX 3.0 is hosted (or a variety of Xen flavors); four gigabit ethernet controllers per blade; CISCO 4948 48 port switch, with 4 ethernets per blade bonded; CISCO 6504e core with Sup-32; Net App 3020 and 3050 for NFS and iSCSI; some EMC Clarion units, likewise.

For CPU we used SPEC CPU 2006 and score about 5-6ish % on VMWare as the same test done on those blades in hard metal. Xen is undiscernably different to the subjective eye than hard metal. I would have to break out large batch testing methodology and run the results through inferential statistics to conclude that there was a difference at all.

I/O is a different story.

The Xen performance claims and the VZ performance claims aren't really useful. They're theoretical. As in, "theoretically, we can stack 100 operating systems on this blade efficiently." Think about that. That's just plain nuts. I can't think of a real use case for that.

BTW, if you like OpenVZ, and have the right use case, the commercial Virtuozzo product ranks as the "best virtualization technology that no one has ever heard of" in my book. They really have their IT management story down pat.

C//

Re:Pointless?

[Wolfrider]

--Running a Linux Squid appliance in a VM, on your Win2k/XP laptop, while browsing wirelessly == Ads blocked + Internet content cached + More secure.

--Testing (Freebsd / PC-BSD / Nexenta / Solaris / Linux) + ZFS + Samba in a VM when you don't have extra hardware to dedicate to it.

--If you're not already a VM-type person, you wouldn't understand.

Re:Linux has inbuilt virtualisation ....

[sglines]

Thanks for pointing to a virtually empty wikipedia article.