Slashdot Mirror
Reasonable Hardware For Home VM Experimentation?
cayenne8 writes "I want to experiment at home with setting up multiple VMs and installing sofware such as Oracle's RAC. While I'm most interested at this time with trying things with Linux and Xen, I'd also like to experiment with things such as VMWare and other applications (Yes, even maybe a windows 'box' in a VM). My main question is, what to try to get for hardware? While I have some money to spend, I don't want to, or need to, be laying out serious bread on server room class hardware. Are there some used boxes, say on eBay to look for? Are there any good solutions for new consumer level hardware that would be strong enough from someone like Dell? I'd be interested in maybe getting some bare bones boxes from NewEgg or TigerDirect even. What kind of box(es) would I need? Would a quad core type processor in one box be enough? Are there cheap blade servers out there I could get and wire up? Is there a relatively cheap shared disk setup I could buy or put together? I'd like to have something big and strong enough to do at least a 3 node Oracle RAC for an example, running ASM, and OCFS."
Xeon based, easy access to multiple drive bays, dual gigabit ethernet, etc. Runs linux, Windows, Mac OS X
...something like this?
I ran into this same situation and found the best cost/performance setup was a Beowulf cluster of netbooks.
You get the cumulative power of those Atom processors and have a huge memory pool to run the VMs within.
Just check the BIOS to make sure that you can set the MB for virtualization.
The simple truth is that interstellar distances will not fit into the human imagination
- Douglas Adams
I ran my first virtual machine on an athlon 2200+ with 768 megs of ram. If it can run windows 7, you can run a VM or 3 (Depending on how heavy you want to get). Essentially take your computer, subtract the cycles and ram required to run the OS and background programs, that's the hardware you have left over to run the os. If the guest OS was compatible with your original hardware, chances are it'll work just fine in the OS.
Is it sad that I am more likely to recognize you and your posts by your sig than your name or UID?
Dell currently have the Studio XPS (2.66Ghz Core i7, 3G RAM, 500G HDD) going for US$800 - for a basic home virtualisation server, it's hard to go past, especially if you spend another US$80 or so to bump the RAM up to 9GB. I can't imagine you could build it yourself for a whole lot less (depending on how you value your time, of course).
(Damn, sometimes I wish I lived in the US. Stuff is just so bloody cheap there.)
64-bit Linux host and as absolutely much memory as you can possibly install.
http://rocknerd.co.uk
I personally use qemu-kvm and im quite happy with it. Thats running on a dual core machine with 2G of ram (probably not enough ram though!).
For the KVM stuff you need have chips which support Intels VT or AMDS AMD-V so your processor is the most important aspect. A quad core would probably be suitable too if you can buy that.
For just experimentation usage its a fantastic alternative to VMWare (I personally got sick of having to recompile the module every time my Kernel got updated).
On my box myself i've had about 6 CentOS VMs running at once but frankly there were not doing much most of the time. Ultimately its going to boil down to how much load you inflict on VMS underneath, my experience with it has not been very load heavy so I could probably stretch to 9vms on my hardware which is probably on the lower end of the consumer range these days.
The most important bits are your CPU and RAM. If your after something low spec you can do dual core 2g ram but you could easily beef that up to quad core 8G RAM to give you something you can throw more at.
Oh and Qemu without KVM is painstakingly slow - I wouldn't suggest it at all.
I currently run VMware Workstation with an Intel Q6600. VMware has a setting to choose to use one or 2 of the cores. Generally, for Linux VMs, one core is enough (unless you decide on GUI). If one goes for Windows Vista/7, 2 is better for performance, but one works okay for XP.
Ram is dirt cheap right now on Newegg as well. I have 8gb of Corsair ddr2 ram I got for 50 dollars after rebates. Non GUI, you can get by with 384-512mb of ram, but otherwise, id go with at least 1024 or more.
The nicer part of VMware Workstation is it now supports Directx 9.0c (but with only shader level 2, still working on 3). Expect a 10 or so perecent in performance droppage though for gaming depending on how many resources you allocate.
Your needs look a bit bigger than mine (mostly trashing VMs and running test software before doing something crazy to the actual box). A bigger CPU such as a Xenon might be more to your liking, since you can have 2 of them for a total of 8 cores (leading to lots of VMs).
My VM server rig is decidedly low-end compared to many I've seen, but it certainly gets the job done. I custom built the box, mostly from components bought on NewEgg; it has a dual-core AMD64 chip (soon to be upgraded to a quad-core), 3 GB RAM, and about 500 GB total drive space between two IDE (yeah, I know, will upgrade to SATA at some point) drives.
The machine runs Ubuntu Server with VMWare Server 2. I can easily run several Debian and Ubuntu VPS nodes on it under light load, and I use it for experimentation with virutal LANs and dedicated purpose VMs. I periodically power up a Windows Server 2003 VM, which uses a lot more resources, but it's still fine for testing purposes.
512 MB RAM, 20 GB disk, 200 GB transfer, five datacenters. $19.95/month.
You can find lots of used servers on eBay that you can mess around with. Sun's v20z servers are pretty cheap and have a decent amount of power.
A lot of the stuff I've run across is rack mounted and keep in mind that rack mounted servers are loud in most cases. So it may not be the best thing to play around with in your home or office.
You don't really need any special CPU to mess around with virtualization, you won't get "full" virtualization but I don't think that will stop you. For more info check out, this page.
I'm currently running a number vm's in my desktop using Sun'x VirtualBox (xVM) whatever they're calling it now. Even within some of the solaris VM's I'm running solaris containers so I'm doing virtualization upon virtualization and my processor doesn't have Virtualization technology support.
If you want to do full virtualization look for server class CPUs. Xeons and Opterons. Using Newegg's power search there is an option to filter by CPU's that support virtualization technology.
If you're primary focus is Oracle RAC, you may want to look at Oracle VM which is Xen based.
Dual Opteron < $600
You can run virtual instances on practically anything. I use VMWare Workstation on an older AMD Athlon 3200+ (the machine on which I'm typing this) and get acceptable performance if I only have one instance booted at a time. You're not going to be playing video-intensive games on the instance, but it'll work find
I maintain a few websites (my blog, a gallery, couple other things) on an old server class machine in the garage. Companies often scrap servers after the 3 year warranty expires, or they've finished depreciating (depending on individual business rules) and they're often fast enough to make reasonable virtual servers. Often you can pick them up at a scrap sale or surplus store, or, if your company has an IT department, get permission to snag a machine that's about to be scrapped.
I recently brought up VMWare's free bare-metal hypervisor ESXi and was surprised at how easy it was to set up and create instances. VMWare has a free Physical-to-Virtual converter you might want to experiment with. It works great with Windows, but is kinda hit-and-miss with Linux.
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
ESX whiteboxing information can be found in a number of places but you might want to start here: http://www.ntpro.nl/blog/archives/338-The-complete-white-box-list.html http://ultimatewhitebox.com/
http://virtualize.wordpress.com/
You can do it "well" on a dual core with 4GB of ram. Even less, but with todays prices you can get a system for a couple hundred if you watch for sales. RAM is you biggest killer that you will notice. Then again, with quad cores with VM assistance going for under $200CDN, thats relatively cheap. If you're worried about HD performance a couple 500GB drives striped will give you over 100MB/s of read speed a relatively small investment.
---- aut viam inveniam aut faciam
we have about 4 machines with 2 quadcore running ESX and about 100 machines (many linux and windows) and 64GB of ram in each esx node... and we have still about 50% of resources free
so grab one quad core machine, with lots of ram (for oracle RAC+ASM+DB you will need at least about 4GB for the 3 RAC nodes, the more the better)
as this is for testing, i would but a plain quadcore PC, with 6 to 8 Gb of ram, install a linux 64bit with xen or vmware esxi
if you have more money, you can buy more ram or even cpu, but you dont really need a blade nor a server, a plain PC will do
ohh i forgot... HD, buy at least 2 HD, to spread the IO load, if you want raid, then you need 4 HD for a raid10... you can also try iscsi with a openfiler based nas/san (another PC, with lot of HDs and several gigabit network card)... of course, the server also need several gigabit network card to increase the IO bandwidth of iscsi
have fun
Higuita
I strongly advise you to do your homework before spending money on non-server class hardware (or before selecting a server for that matter). VMWare runs on a lot of hardware, but it also fails badly on a lot of consumer grade motherboards. There are some list (White Box Hardware Lists such as http://ultimatewhitebox.com/) you can check. After spending some time on name server HW and on White Box gear, I can tell you that the name server gear is a lot more compatible, easier to work with, and worth the money (if you have it). If you are doing casual stuff and don't mind the considerable pain you will have to go through to get patches and select disks systems and other components, consumer gear will let you play a bit. As for doing anything serious with more than one VM on a box - not likely. Xen is a commitment, as is VMWare or any other VM system. It is going to eat the box if you do anything other than dabble in it, and you are going to spend some real money if you intend to do much with VMWare (think $3K - $5K to get very serious). Running a VM is easy. Running multiple servers, backups, external disk systems, etc. is real work and costs real money for all the extra stuff you will need. If you stick to Linux you can save a bunch, but if you intend to do any real work with MS Servers, you are going to need several licenses, and iSCSI targets, back up tools, etc...You won't actually learn too much before you go to that level that you can't learn with VMWare Workstation (a great product but not anything like a production server environment). You can get you feet wet for nothing but time with most of these tools, but you can't get real, in depth experience with what it takes to run a production cluster, replication, remote storage, live replication, and all the rest of the things you need for real production unless you actually set up a production like system - that means real servers (White Box or name brand) and lots of hardware. You won't be able to see much with less than 8 cores, and 16GB plus some local RAID and iSCSI network targets. You can get started, but if you are going to spend money, I really think you should start to buy gear that is going to build towards a real server environment or you should stick to home systems and maybe run VMWare Workstation or some other stand alone VM just to play with it. VM Mode Linux (not very popular today) or some Xen sets for personal use would give you some understanding of VM concepts, but not a lot of basis for real production issues (at least they did not for me and I was a pretty heavy development user). Production VM deployments have a lot of issues that all take real in depth study, and lots of resources (iron) to get right. On the other hand, you can get a Supermicro, a Dell, or HP server with dual Xenon quad cores for less than $4000 with some nice disk. Get 4 or 5 containers under a VM and set up a replication to another server and a remote iSCSI disk and then you have enough to start to actually do real learning on. Of course the license fees will be way more than the hardware costs, if you are using MS tools and VMWare. ESXi is OK but unless you are going to go deep and do it all the hard way (hack the OS) you can't do a lot with the free version. With Xen, if all you want is to run a couple versions of Linux, just get a quad core box and have some fun...it doesn't really give you much production knowledge, but you will have some interesting test you can try. What I am really saying is - with only 4 cores you can do some useful things to support development,and you might make a nice personal server for you private web sites, but you don't have enough iron to experience the real issues of production VM management. If you are going past what a developer does (or a tester) and looking at a operations type environment you will need 8 to 16 cores on multiple boxes. This is a lot more than a home user typically wants to spend. IMO you also can't really expect to be really good on more than one system unless you do it day in a
Amazon EC2 is what I use for stuff like this, both windows and linux boxes everything available at a single push of a button. I
also use it alot for development, fire up a machine load and go.
Got Code?
Reading 'cayenne8', I can't help but imagine a V8 Porsche, and because I'm a car guy, for good or bad, this shifts the focus of my comment toward resources, specifically what is available, versus what is acceptable or tolerable.
Let's say you're a one-man Lab, incorporating all the SA, Developer, and Midware functions into your 'play' with this environment. How much time will each environment spend heavily plowing into loads?
If your intent is to deploy RAC in a multitude of scenarios, in short order, with a minimum of intervention, you may be able to get away with $1500 to $2500 worth of NewEgg parts (think high throughput - RAID, Max. RAM, Short access times, etc.) and the virtualization technology of your choice. Personally, I find VirtualBox capable of everything I need as far as virtualization and deployment goes, however, you need to be able to leverage 'fencing', with likely puts you into VMWare territory.
Fortunately, VMWare Server is 'free', and CentOS and OpenSuSE support some of the more advanced features of HA on Linux. Then again, if we're looking at resources as a major factor, then Redhat and Novell might be worth looking at, as they both offer 60 to 90-day evaluation licenses for their Enterprise Linux products, which may offer a prettier and more 'honest' (per the documentation and common expectations) implementation of their respective HA features than the freely-available, and in some cases, in-flux versions of the same software.
As far as RAC goes, take a look at the requirements for RAC, per Oracle's installation guidelines,, and size/spec from there. I believe you can get away with 16GB - total, if you have the capability to size the VM's memory access, or otherwise configure the amount of addressable memory, or put uo with or hack Oracle's RAC installation pre-flight. There is also valuable documentation available on your chosen OS vendor's sit, which may even be Oracle, who knows..
You may be hell-bent on performance, however, and you may be looking for the ultimate grasp of technological perfection, as it exists at Sun Mar 22 17:29:59 EDT 2009. In this case, you may want to look at Xen, which is available on Solaris as their 'xVM' technology, as well as on various Linuxes and BSDs.
On the other hand, you may be a Mac guy, with a decked-out Octo-core Xeon Mac Pro, where you have the option of Parallels and Virtual PC and something else, in addition to Sun's VirtualBox mentioned above.
Ultimately, things to keep in mind may be shared disk requirements, fencing options, and VM disk and memory access.
YMMV
You can do Oracle with just a single machine running multiple VMs; however, if you really want to get serious, you should consider building two physical machines. One each machine, create a virtual or two with 1-2G of RAM. for the shared disk, use DRBD volumes between the two.
My test RAC cluster has two AMD X2 64-bit systems with two gigabit NICs each. CompUSA has a similar machine for about $212 on sale this week. Newegg prices are similar. You'll need to add a couple extra Gig NIC and some more storage. Still should cost under $400 each.
On each physical system I used CentOS 5.2 with Xen. I created LVMs on the physical machines as the root volumes. Also carved out a separate volume to back the shared volume. Then I carved out a xen virtual machine on each with 1.5G each. I put the DRBD network on one pair of NICs. The other pair was used for the network and heartbeat (virtual ethernet devices).
The difference between 'server class' hardware and you beige box PC is that the more expensive 'server' is a lot more reliable and has extra remote access and hardware monitoring features. That's about it. If all you want is to run virtual machines in a test environment, just get a desktop with a hefty CPU and a whole whack of RAM and you're set. A good 'gaming' machine without the video card would be fine. You don't need to spend extra for a 'server'.
You say you want to go "cheap", that you don't want to spent too much money, yadda yadda... and then you go on to mention things like "cheap" shared disk and "cheap" blade servers?
What you realistically need and want are two different things.
I'd suggest a cheap quad-core AMD Phenom II system with 8G or so of RAM. Nothing too fancy. that I assume you're going to be running a Windows host OS, or VMWare ESX. More RAM will be needed for the Windows host OS, obviously.
Absolute lowest-end hardware you'd want to look at getting is an AMD Athlon 64 x2 or Intel Core (IIRC) based system. In other words, you want/need the VT support, or it'll be purely an emulated environment, and substantially slower than native (30%?), not just marginally (10%?).
I recommend AMD hardware because it's got a better price/performance point, and because unlike the other stuff in the "reasonable midprice" range for Intel, it's got the memory controller/north bridge integrated into the CPU (for newer gen stuff). I'd say go Phenom or Phenom II without any hesitation.
With a CPU like this, there's no reason you couldn't build a full system for around $450-500, sans storage. You could probably find a suitable "starter"/deal system for $300 from TigerDirect that'll do the job just fine with a little more RAM and another drive.
For disk, just go with an SATA RAID card (LSI are good) and 3 1Tb disks. That's about as cheap as you'll get and still have room to work.
~/ssh slashdot.org ssh: connect to host slashdot.org port 22: too many beers
The main requirement is enough RAM for two operating systems plus some extra for the virtualization system. The CPU is less important. I run Windows XP Pro as a virtual system on a Linux host with VMware Workstation 6. It is a 5-year-old Athlon 3000+ box with 1 GB of RAM. I allocate 512 MB to Windows, which is about the minimum for XP. Current Linux distributions need at least 256 MB and VMware is something of a memory hog itself so 1 GB is about the minimum RAM for this setup. Windows is perhaps just a smidgen slower than it would be if running natively on the same hardware but the difference is minimal. It does not have much effect on the speed of Linux apps running simultaneously. Things bog down fast if you try to run more than one virtual system simultaneously but VMware is good at using multiple processors for this. I did some work which involved running up to 6 instances of FreeBSD simultaneously on an 8-core Xeon system with 4 GB RAM. Up to 6 it did not slow down much. Over 6 it got sludgy. Have fun! Mike
The biggest thing that you have to watch out for with VMWare ESXi is the hardware compatibility list. You will run into trouble with two major components: RAID controllers and network adapters.
The network adapter solution is simple: buy the most plain-jane Intel PCI or PCIe adapter that you can find. Examples of ones that are known to work right out of the box are the Intel PWLA8391 GT (single-port PCI) and the Intel EXPI9402PT (dual-port PCIe). I own both of these and can personally confirm operation with the latest version of VMWare ESXi.
The drive controller situation is both complicated and -- more importantly -- expensive. Overall, Adaptec seems to be the most well-supported controller hardware out there. I have tried LSI controllers, but they often don't play well with desktop boards. Unfortunately for experimenters, the built-in RAID on practically every Intel motherboard is completely unsupported in RAID mode. Obviously no enterprise environment would be using on-board RAID like that, but it would be nice to have for experimentation.
Which brings me to my favorite storage solution for ESXi: Openfiler. Openfiler is an open-source NAS/SAN solution based on rPath Linux. It turns any supported PC into a storage applicance, and can share its storage in a plethora of ways. In the case of a virtualization effort, it has two major things going for it: it supports any storage controller that Linux supports, and it supports iSCSI and NFS.
If, say, you do have a machine sitting there with Intel on-board RAID, you can install Openfiler there. While the hardware might not work under ESXi, it'll work great for Openfiler. Even better, Openfiler also supports Linux software RAID which can be superior when it comes to disaster recovery (no need to have a specific controller card to see your data). With this in mind, you'll be able to get Openfiler running on just about any hunk of shit box you have sitting around.
Once you have Openfiler set up, you can take the next step in virtualization-on-the-cheap: installing ESXi on a USB flash drive. There are a number of tutorials on the web for this (just google 'ESXi USB flash install'), but the basic process amounts to extracting the drive image from the ESXi installation archive and simply writing it to flash with dd (on Linux) or physdiskwrite (on Windows). Once this is done, you can plug the flash drive into nearly *any* recent x86 hardware and it will boot ESXi. A really neat feature that you get along with this is the ability to substitute hardware with ease, and upgrade to later versions of ESXi simply by swapping the flash drive.
Once you have ESXi installed, create an iSCSI volume on your Openfiler box. Then, use the VMWare management software to connect the ESXi box to your Openfiler iSCSI volume. You can then create virtual disks and machines from the actual USB-flash-booted VMWare host, all of which will be stored on your Openfiler machine. You may also want to try experimenting with NFS instead of iSCSI. There are a couple proponents of this out there that say under certain circumstances it's even faster than iSCSI. It also makes backing up your virtual machines a little simpler since an NFS share is generally easier to get to than iSCSI from most machines. Another cool aspect of the Openfiler-based configuration is that you will get access to another whiz-bang feature of VMWare called vMotion. Since the VMs and their disks are stored centrally, you can actually move the VM execution from one ESXi box to another - on the fly.
In all, this is a great way to get your feet wet in virtualization because you can have a pretty sophisticated setup with very basic commodity hardware. If you want to go the extra mile and get really fancy, put a dedicated gigabit NIC (or two, bonded) in each box and enable jumbo frames; the SAN will be more than fast enough most anything you'd like to do.
Good luck!
Well you don't clearly state what you wish to accomplish nor how much money you have so it is hard to answer. But maybe such setup will be OK.
Build yourself custom PCs.
Storage server:
- good and big enclosure which can fit large ammount of drives
- moderate 64bit AMD processor (really any - you will not be doing any serious processing on storage server)
- any ammount of RAM (really 1 or 2 gigs will be enough)
- mobo with good SATA AHCI support (for RAID) and NIC (any - for management) onboard
- one 1Gb PCI-* NIC with two ports
- 6x SATA2 NCQ HDD (any size you need) dedicated for working in RAID - software based (dmraid) RAID1+0 array configuration
Virtualization servers (2 or more):
- you need the virtualization servers to have the same config
- any decent enclosure you can get
- the fastest 64bit AMD processor you can get preferably tri or quad core (it will do the processing for guests) with VT extensions
- as much RAM as you can get/fit into the machine
- mobo with VT support, one (any - for management) NIC onboard
- one 1Gb PCI-* NIC with two ports
- one moderate SATA disk for local storage (you will be using it just to boot the hypervisor) or disk-on-chip module
Network switch and cables:
- any managed 1Gb switch with VLAN and EtherChannel support, HP are quite good and not as expensive as Cisco
- good CAT6 FTP patchcords
General notes for hardware: :)
- make sure all of the PC hardware is *well* supported by Linux since you will be using Linux
- if you can get better (quality wise) components, good enclosures, power supplies, drives etc. - since it is a semi server setup you don't like it to fail for some stupid reason
Network setup:
- make two VLANS - one for storage, other for management
- plug onboard NICs into management VLAN
- plug HBA NICs into storage VLAN
- configure ports for EtherChannel and use bonding on your machines for greater throughput
Software used:
- for storage server just use Linux
- for virtualization servers use Citrix XenServer5 (it is free, has nice management options, supports shared storage and live motion) or vanilla Xen on Linux, don't bother with VMWare Server, VMware ESX and Microsoft solutions are expensive
Storage server setup:
- install any Linux distro you like (CentOS would not be a bad choice)
- use 64bit version
- use dmraid for RAID and LVM for volume management
- share your storage via iSCSI (iSCSI Enterprise Target is in my opinion best choice)
Virtualization servers setup:
- install XenServer5 (or any distro with Xen - CentOS won't be bad)
- use interface bonding
- dont use local storage for VMs - use storage network instead
Well here it is. Quite powerfull and cheap virtualization solution for you.
I run a VPS hosting company, my job is to research, setup, and maintain a cluster/grid of servers running Xen with hundreds of guests (virtual machines). For testing and even for deployment, we've used machines as simple as a single-core AMD 3800 with 80GB disks in RAID-1, and 1GB of RAM. These aren't the most profitable machines, as they can only support as many virtual machines as can pay for the electricity and square footage, but they work perfectly fine for up to approximately 12 guests. I do highly recommend a dual-processor or dual-core system, though.
If you want to know how much you can stress a system, for highly-dense numbers of guests, I try not to load more than 15 guests and 2GB of RAM per CPU core. Of course, if you plan to have a low-density of guests (say one guest per core), you'll need to adjust accordingly.
I found that for my home office, where I often have pretty excessive needs such as installing multiple operating systems and performing multiple large compiles at the same time, a dual quad-core system with 16GB of RAM is overkill. Right now, I'm using a single quad-core workstation with 8GB of RAM and it works pretty well for me, and is probably still a bit more than I need.
The biggest problem I see with those getting into virtualization is that they think that virtualizing things makes them magically need fewer resources.
You can share CPU time as most apps will not drive the CPU 100%, having said that it is often best to have as many cores as you can afford.
Do not over allocate your RAM, if you can have as much ram as needed for how much you allocate to the VMs, if you over lap you will get a huge performance hit.
Sparse disk is a fairly new feature only in some VM systems, you will need lots of disk for all of the VMs, also you will probably want to run them on different LUNs or disk groups so you don't get lots of thrashing on the drives.
If you are only running 1 or 2 VMs as a test then really all you need is to up the ram a little and make sure the host meets the minimum specs of the VM applications.
EA David Gardner -"... but the consumers have proven that actually what they want is fun."
I just did this myself. I ended up just shooting for cheap hardware on the theory that if it breaks in 2 years I can just replace it. I have a Quad Core Phernom with 8 gig of RAM and two 750gig drives. Chucked VMWare on it and havent had any issues running about 8 or so VM's on it. It also serves up media using TVersity and is a network share dump as well.
The biggest issue I have had so far, is Disk Driver perfomance. If you are planning on running multiple concurrent VM's then go for as many HDD's as you can. Stick the most load intensive ones on seperate drives and you will really see the benefits.
I was running VM's back in the days of DOS. First with Taskview and later with Deskview running 4 concurrent DOS v5 sessions on a single-core 8088! And if they slowed down I'd just push the turbo button and go from 4.77Mhz to 8Mhz! oooWEEEE! That's right! And I'd tote that 45lbs IBM-XT all the way to school...in the snow! And I LIKED IT!
The main thing you need for VMs is memory. There isn't really any good way for VMs to share memory, they each need their own. So decide what you want to give each system, and make sure you've got that much on the host plus like 1GB for the host OS and VM software. Good news is RAM is cheap. You should be able to pick up plenty for not much money. If you get a system based on a 975X or P35 chipset, you should be able to drop 8GB of RAM in it. Ought to be more than plenty. Those are cheap and plentiful these days too. Plus, they use DDR2 RAM, which is currently the cheapest. An Intel DP35DP motherboard might be a good choice.
As for a processor, kinda depends on how hard the VMs will be working. That they can share. So if they are mostly sitting idle, like say a web server serving up static pages, you can get away with not a whole lot of CPU power. If you want them all to be working all the time, you need more. A Core 2 Duo will probably do just fine if you that's what you've got or you need to keep the cost as low as possible. However, this is a case where a quad core would make more sense so that's a good way to go if you can. Goes double if it's the same price. Like say you can get a 2.4GHz Core 2 Quad for the same price as a 3.0GHz Core 2 Duo. While for a desktop you'd probalby want the duo, get the quad in this case. Might look at the Q6600 or Q8300. Both are under $200 and would do a real nice job. Note that the Q8300 is going to need a P35 board, teh 6600 will work on a 975 board.
Disks are a real big "it depends." VMs can be set to grow as they need more space, and so you can in theory have a bunch of VMs sharing one small disk, along with the OS. However, that can lead to performance problems. Harddrives suck at random access, and if a bunch of VMs get going on it at the same time, that's what you'll get. So ideally you'd have one VM per harddisk. In reality, that's probably overkill unless you've got lots of disks laying around. However if your VMs will be heavy disk access, you might want to consider getting 2 drives for them since drives are cheap. Either way, the best idea is to have the preallocate all the space they need for their virtual drives. You get better performance that way, even though it wastes drive space, but again, drives are cheap. Maybe start off with one drive for the VMs and if you find they are getting bogged down, buy another and move them over. They are just files on the drive so easy to move.
Those are the biggest factors to think about. You get a quad core, good amount of RAM, and enough disk space, you should get great performance. If you need to save money, don't feel like a dual core won't work fine. Really the only thing not to cheap out on is RAM. You need to have enough, virtual memory is WAY too slow. So if you want 4 VMs with 1GB each, have not less than 5GB in the system.
Supposing you do have plenty of cash and want to further increase performance one other thing you can look at is NICs. VMs don't do a great job of sharing NICs presently. VMWare is actually working on that, but right now you get ideal performance with one NIC per VM. Not normally a big deal but if your VMs do lots of traffic it can matter. So if you want, get more NICs. One of those multi-port NIC cards works just as well. This really isn't all that necessary, but you can do it if you are after the best performance.
I'm doing this now, running a company infrastructure on Xen3.2 and 2 non-server class machines. These are Gigabyte and MSI Core 2 Duo motherboards running at 2.6 and 3Ghz. Each with 4GB of RAM, dual GE NICs, RAID1 drives. Nothing special.
Application Systems are:
- enterprise email/calendaring/IM
- CRM
- document management, file/print
- project management
- VPN
- internal website / wiki
- VoIP/PBX
- Monitoring, PKI to manage VPN credentials
- LDAP for authentication across all these systems.
Each VM can be migrated to the alternate host-server with minimal downtime (sub 1-second).
Backups are rdiff-backup based - complete VM backups take less than 2 minutes. Most of the machines are only 10GB disk images. DMS is 20GB since we're a document heavy enterprise.
Total CPU is hardly ever over 20% utilized, basically, only during backups. Because Linux grabs available RAM for disk buffers, it is all used, but everything easily fits on a single 4GB RAM box with excellent performance. This is nice so system upgrades don't impact running systems, but most of the work can still be performed during work hours. Having 2 boxes lets me perform system upgrades without any risk to the "production" system.
I'm running 64-bit Ubuntu for all.
I tried VMware - it wouldn't load ESXi on my hardware and VMware server is "too heavy." For some of out customers, VMware is the best solution, but they are Windows shops.
I use VirtualBox on a laptop, but it isn't ready for enterprise use. Another year or so and it will be stable enough. 3 of my partners also use VBox on their laptops. It's easier to setup a VM with Linux than fighting with cygwin.
Xen and Linux go together nicely. I plan to bring up a few Windows VMs - I've read they work fine under Xen3.2, but haven't had time to try them yet.
I've blogged about most of what I've learned along the way. Learning about systems, applications, Xen, and other virtualization issues.
Memory, and lots of it. Nothing else will help as much for running multiple VMs.
Memory is dirt cheap, I recently bought 8 gigs of ECC ram for ~100 USD. Of course, over 3-4 gigs, and you need a 64 bit OS, I use Ubuntu 64, but I know others who use Vista 64 to good effect.
At least 2 cores, 3 or for doesn't hurt either. There's great value in both AMD and Intel at the moment, Intel owns the top end, but at the low end or midrange AMD tends to have the better value.
If possible, get a separate drive for at least your main OS, and run the VMs off their own drive. More spindles == more IO, I run 6 drives in my box, one for the OS, and 4 raid 5 for my homedir for speed, capacity, and safety, and one drive bay I swap out for a spare I keep offsite that holds my backups. Linux software raid is great for this use, and with modern multi-core processors you won't notice the overhead.
If you can only afford maxing out one thing though, make it the memory.
Blessed are the pessimists, for they have made backups.
The basics to virtualization comes down to the # of cores, the amount of memory and the # of spindles, though if you've read through the latest reviews of SSD's on Anandtech you can replace spindles with Intel X-25m or X-25e drives. In a virtual environment random reads/writes are FAR more common than any sequential read/write access, therefore you either want a high spindle count or fast SSD drives, depending on your budget.
A quad-core system with 4-8 gb or more (depends all on how much memory you want to allocate per VM) and either alot of SATA drives or one or more Intel SSD's will be very fast.
Most VM's that I've seen miss the boat on the disk throughput but deliver on the amount of memory and RAM. Given those requirements you can find a quad-core processor + mb for ~$200 (I bought mine for 180), 8 GB RAM (PC2-6400) ~$60-80, and one intel ssd for $325. You should be extremely happy with the performance. If you need storage, I'd suggest buying some sata drives in addition.
Basically, as long as each virtual node isn't doing any WORK, you don't need any special hardware. And even if they are doing some work, but just not a lot. We have 5 Linux Xen VMs in production on a 1600Mhz Celeron with 768MB of RAM, works fine, no problems.
The CPU is almost irrelevant - you'll need whatever CPU you'd need to do all the things you're doing, plus some overhead, but it's not like it falls apart.
RAM is the only critical thing. You need at least 96 MB for the host and 24MB for each additional live Xen VM, as I recall (That's probably not precisely right) But you'll naturally be swapping a ton if you do that. A more reasonable VM has 128M - 256MB of RAM itself, so you need that for each active VM. But again, that's only for each one running at a time.
Or if you are going to swap a bunch, get better disks :)
In any case, I definitely wouldn't climb the price curve of equipment to do this; don't buy anything on the bleeding edge - look at arstechnica and just max the RAM on a value box - or maybe upgrade the MB to something that takes more RAM.
Used, commodity computer equipment is usually not price effective compared to the cheap end of what's still available new. But pay attention to the price point where it's cheaper to get (and power, while they're on) TWO value boxes than to pump up the one box you've been thinking of higher.
Looking for freelance Actionscript (Flash/Flex) or ColdFusion work and/or freelance developers. Email me, put Slashdot