Desperately Seeking Xen

AlexGr sends us to an excellent article on the state of Xen by Jeff Gould (Peerstone Research). He concludes that the virtualization technology has some maturing to do and will face increasing competition for the privilege of taking on VMWare. Quoting: "What's going on with Xen, the open source hypervisor that was supposed to give VMware a run for its money? I can't remember how many IT trade press articles, blog posts and vendor white papers I've read about Xen in the last few years... The vast majority of those articles — including a few I've written myself — take it as an article of faith that Xen's paravirtualizing technical approach and open source business model are inherently superior to the closed source alternatives from VMware or Microsoft."

Comments

Re:Need a special processor

[Mostly a lurker]

It is true that Xen requires special hardware to legally run MS Windows. It is also better for performance, generally, to have such hardware. However, there is nothing stopping you from running Xen on pretty much any computer you are likely to own as long as the VMs are Linux based.

Re:vm ware

[Per Wigren]

If you want to run multiple linux instances on the cheap then xen is the way to go at the moment. Except that OpenVZ is a better way to go in that case. If you are only going to run multiple instances of Linux, with OpenVZ you don't need to preallocate a fixed amount of memory for each VM, the root filesystem can be a subdirectory of the root OS instance's filesystem, among many other things. It can do just about everything that XEN can do, including live migration to other physical nodes.

Re:what went wrong is

[stevey]

Not true.

If you have VT-capable hardware then you can run Windows under Xen. You do need the hardware to support it though, and that is a problem for some home users. Recent AMD and Intel chips have slightly differing VT support but both work.

I run Xen at home along with xen-tools (which I wrote) to easily create new Debian guests on demand. These are used for software testing, hacking, and general service isolation.

I think Xen is just now reaching "mainstream" in the sense that you don't have to be an early adoptor or major tinkerer to get it working. Now that distributions are including Xen kernels in their newer releases it really us available for all.

Using it...

[dmayle]

You can choose to believe the hype or not, as you wish, but I'm using Xen in my production environment, and it's simply fantastic. I've got friends with companies who are doing it as well, and it really changes how you think about administration.

Of course, there are some learning curves. For example, how you manage 3-7 servers is completely different from how you manage 20-30, even if they are all virtual. There's a lot more emphasis on system images, isolating functionality, reproducing configurations. On the other hand, dev environments are so much easier to build-up and tear down.

I just wish the OpenBSD port was in a usable state. The mercurial servers hosting it are often down, and even when they're up, I haven't been able to get a working kernel compiled from the sources (even after doing some of my own bugfixes). And last I saw on the Xen lists, Christoph Egger (the guy doing the OpenBSD port) submitted a security patch related to stack slamming, and the Xen guys were kind of like, "meh, security's not really a priority..."... Oh well, here's to keeping my fingers crossed

The Problem With Xen

[Aeonite]

Is that Gordon Freeman ruined it.

Xen "Just Works" (I know. I use it every day)

[eno2001]

Not only that, but I've been running it in a production environment for about a year and I'm about to deploy a HUGE set of servers as VMs using it. Xen beats VMware in one arena: price. If you use the open source version (which I'm doing) it's free. Only VMWare's ESX can compare to Xen. And unlike some people here have been saying, you DON'T need a special processor for Xen unless you plan to virtualize Windows. In my environment, I'm only virtualizing Linux, so I can use regular x86 CPUs dating back to 1998 for Xen. The only exception is the deployment of Zimbra I'm going to do. It requires Redhat Enterprise Linux 4 and NPTL, so I can't run it paravirtualized, it must run HVM which requires the special processors. However, who today isn't getting new hardware with HVM support?

Currently my two Xen servers here at work serve out about four VMs (all paravirtualized on older hardware) for critical and I/O intensive tasks like proxy servers for nearly 1000 machines, or the firewall syslog server for a dual T3 link with about 5000 users behind it sucking the bandwidth dry. So you can't claim it doesn't perform either. Now, if you want point and click administration and an easy set up, then yeah, Xen is behind the times. But performance wise it's leaps and bounds above VMWare. Trust me, I was a VMWare fan before you were in virtualization diapers. And I still am for some applications. But for places where I need something to be cost effective AND give me the features of VMWare ESX, Xen is the ONLY answer.

Lies, all lies

[C_Kode]

Another question hanging over Xen performance concerns the availability of paravirtualized drivers for Windows.

This isn't true completely. The problem is you cannot get these drivers by downloading the OpenSource Xen. You MUST buy the XenSource version. If you run Windows on the *complete* open source version, your network throughput is going to suck like you would not believe. You have to use the XenSource version to get the paravirtualized drivers that bring the network performance closer to what it should be. Virtual Iron has a set of drivers also. (which I believe are better than Xen's, but don't hold me to that)

I found a lot of great insight about virtualizing from Xen to VMWare to Virtual Iron and others on this site. http://ian.blenke.com/xen

Re:vm ware

[gtrubetskoy]

Except that OpenVZ is a better way to go in that case. For fairness it should be mentioned that aside from OpenVZ there is also Linux VServer which does a few things better than OpenVZ (though OpenVZ does some things it does not). Our preference has always been VServer, it's a well-run project with emphasis on quality and well thought through design rather than quantity.

Xen (and virtualization) is for the Enterprise

[xzvf]

While it might be nice if all these things are easy and work well for the hobby crowd, the real money in virtualization is in the enterprise space. Most servers in enterprise environments run 15% max and are refreshed every 3-5 years. The special processor matters less in that case, and the competition is between a mature VMWareESX server (not free), a hardware based IBM and Xen. Microsoft is a surprisingly minior player. VMWareESX server is very good for x86 consolidation and saves customers money, but is very expensive. It is still the best option for Intel based consolidation. Xen has deep penatration in enterprise lab environments. It is just getting the enterprise management tools to move into real production. IBM is very good at virtualization and stability, but on proprietary power and mainframe hardware. Xen will be fine, because the market is very immature, but expect more seamless and non-attrusive virtualization on the desktop.

Guaranteed Results

[catdevnull]

If you want to get a colorful thread of comments started on slashdot, there are 3 ways to do it with guaranteed results:

1) Say something bad about linux (or about Apple).

2) Say something good about Microsoft (or about Apple).

3) Throw a grenade in the room about Open Source software like this:

The vast majority of those articles -- including a few I've written myself -- take it as an article of faith that Xen's paravirtualizing technical approach and open source business model are inherently superior to the closed source alternatives from VMware or Microsoft.

I'm not making any value judgements here--I'm just amused.

Re:Xen "Just Works" (I know. I use it every day)

[div_2n]

The only exception is the deployment of Zimbra I'm going to do. It requires Redhat Enterprise Linux 4 and NPTL

Last I checked, Zimbra runs on Ubuntu 6 just fine.

Re:Xen "Just Works" (I know. I use it every day)

[eno2001]

It depends on what I'm doing. If you weren't trying to be cute, I'd say you were trolling. In reality, it's very common practice to use LVM to clone a filesystem, make some changes to the various files that set IP and hostname as well as other unique host settings and bring up alternate "Test" VMs on a Xen box. So some days I might be running three VMs other days eight or ten. It all depends on what I need to do.

As an aside, I forgot to mention that there are NO other products other than VMWare ESX that offer "live migration" of a running VM from one hardware host to another. That's right... you can take a VM that is running with many users actively using it and move it from one physical box to another with only a few milliseconds down time. The users NEVER notice. The free VMWare server can't do that. Micrsoft's Virtual Server can't do that until they have a hypervisor. And there really isn't anything else that can.

Xen's Maturity

[Krondor]

He concludes that the virtualization technology has some maturing to do...

I RTFA and it says very little about the maturity of the actual Xen technology. The article is more a point about several non-related factors;

1.) There is a lack of pretty management interfaces.

True, but these are in the works from Red Hat, Novell, XenSource, and various other ends. Already some of them look pretty promising, but if you are a real admin you don't need them in the first place. There is nothing wrong with using the command line tools to manage your Xen virtual guest environment.

2.) There is a lack of references for companies using Xen.

How does this relate to the viability of the Xen virtualization? Yeah it makes management feel nice and fuzzy that others are using something, but this does not relate to how well the Xen technology performs. I also suspect that like many open source projects, there are many people using it that do not report it. Novell has personally contacted me and my company to ask us to assist in their new paravirtualized Windows drivers initiative and then be a reference for the technology. It seems that at least some companies are moving to address this, at any rate.

3.) There aren't many benchmarks about Xen versus VMWare.

VMWare does not allow benchmarks they do not approve of. It's in that draconian EULA you agreed to by using it.

4.) It's awkward to paravirtualize Windows.

Yes, it is. Novell signed the soul sapping agreement with MS and as such is pushing some paravirtualized drivers for Windows. The article continually talks about woes with Xen on Red Hat. Red Hat didn't sign the agreement and will require some much more intelligent coding to make this happen. It might never happen, so for Windows it's full virtualization with VT (or AMD's equivalent) or bust. Sorry, use SUSE for it or use full virtualization. It's an MS issue not a Xen issue.

5.) MS's new Viridan Virtualization Platform is using paravirtualization as well.

Yep, that should be a testament to the approach versus VMWare. Though it is interesting that VMWare now has a Linux kernel virtualization implementation similar to KVM. It seems VMWare is headed to paravirtualization as well. Obviously Xen did something right.

6.) There is a lot of competition.

True. How again is this relating to Xen as a virtualization technology.

Again, I'm not saying Xen is perfect. It definitely has issues and room to grow. I'm just saying that the article makes little, if any, relevant points to Xen's virtualization technology.

Data Center USA

[fyoder]

I stopped reading the article with this quote:

Oh my. Editable XML configuration files, obscure command line interfaces, grayed out options in the GUI? Thanks, but no thanks. This thing doesn't sound like it's ready for prime time in Data Center USA.

Are sysadmins at "Data Center USA" morons? "Oh nooo, command line time, I hate that. Oh nooo, my option I want is all grayed out! Help me, help me! Oh I am so sad now."

Deploying vm stuff is not the same as using a word processor. "Data Center USA" is in real trouble if their sysadmins aren't any smarter than regular desktop users.

Re:Timely for me!

[TheRaven64]

There was some interesting work presented at the XenSummit which should be making it into the main tree eventually on 3D support in Windows. The idea was that the memory layout was adjusted slightly so that Xen and dom0 lived nearer the top, and Windows in an HVM domain lived at the bottom. This allowed Windows to use existing 3D drivers, without having to do any address translation when performing DMAs (not an issue if you have an IOMMU, but hardly anyone does yet).

The problem with giving access to hardware to guests at the moment is that without an IOMMU, any DMA request the driver issues will read or write memory from a physical address indicated by the driver. In a virtual machine, what the driver thinks is a physical address is actually a virtual address. This means a DMA request will read from or write to an arbitrary memory location. By putting the HVM guest at the start of memory, this translation is the identity function, so the driver will work. The only downside is that you lose protection from other domains; a malicious driver can still damage your other VMs or even the hypervisor.

Re:Fundamental performance issues

[Courageous]

20 Dell 1955 Blades; 16G ram; 70GB SAS 10K drive (one) on which ESX 3.0 is hosted (or a variety of Xen flavors); four gigabit ethernet controllers per blade; CISCO 4948 48 port switch, with 4 ethernets per blade bonded; CISCO 6504e core with Sup-32; Net App 3020 and 3050 for NFS and iSCSI; some EMC Clarion units, likewise.

For CPU we used SPEC CPU 2006 and score about 5-6ish % on VMWare as the same test done on those blades in hard metal. Xen is undiscernably different to the subjective eye than hard metal. I would have to break out large batch testing methodology and run the results through inferential statistics to conclude that there was a difference at all.

I/O is a different story.

The Xen performance claims and the VZ performance claims aren't really useful. They're theoretical. As in, "theoretically, we can stack 100 operating systems on this blade efficiently." Think about that. That's just plain nuts. I can't think of a real use case for that.

BTW, if you like OpenVZ, and have the right use case, the commercial Virtuozzo product ranks as the "best virtualization technology that no one has ever heard of" in my book. They really have their IT management story down pat.

C//