Slashdot Mirror
Ask Slashdot: Computer Test Lab Set-Up For Home?
An anonymous reader writes "For as long as I've been playing around with computers I've had a home test lab. I found it to be a great learning tool. However, I haven't invested much money into it lately and because of aging hardware I can't get what I want out of it anymore. So a revamp is in order. I've looked into several cloud vendors for a box I can rent to do some virtualization, but it doesn't seem to be cost effective or practical. What are your thoughts on it? What set-up do you have at home for tinkering? Have you looked into hosted solutions for this?"
Buy a computer.. Put >8GB of ram in it (i would recommend 2GB per VM, and 2GB for the host). Maybe some nice fast disks..
Load VMWare ESXi, or another OS and virtual machine software of your choice..
The ability to snapshot and restore things will save you so much time testing things, you'll wonder how you used to get things done before. Maybe, setup a second system or laptop for things like wireless testing, drivers, etc.. things you can't simulate in a VM.. but with the virtual networks in most VM's, you can setup some very, very complex networks...
What are we going to do tonight Brain?
I'm guessing Virtualbox (http://virtualbox.org) is somehow not viable? Have one nice computer, and run quite a few virtual machines running on it?
It is not entirely clear what you want to tinker with. What do you want to test? Are you wanting to tinker with hardware? Using different software? Writing software? If the latter, what kind? To what end? This is a useless summary of your question.
What kind of 'test lab'?
What software will these be running, and what kind of hardware will they require?
What are you wanting to do with them? Run a neural network, run cluster computing?
Too many details left out to even begin.
Get a decent PC with, say, 4GB of RAM
Install VMware ESXi (free) and tinker with virtual machines all day long
-- / .- / - / -
Get a box with sufficient resources to run xen, kvm, vmware, virtualbox, or whatever virtualization product meets your need. Most support virtual networking and vlans if you need to hook up routers and equipment that can not be virtualized.
Would imagine it depends very heavily on what you are actually doing
I’ve got:
- A powerful desktop,
- Large fairly expensive file server and a cheap backup file server (same capacity, but cheap hardware and drives)
- Several old boxes (mostly previous desktops and stuff I rescued from people who were going to throw them out) one of which is acting as a virtual machine host..
- Two intel atom based boxes. One I use for a whole bunch of random stuff (for instance, all the various UPSen I have are plugged into it, and it coordinates a shutdown of everything when any of them run low on battery. The other I use as my hardware tinkering box.. when playing with stuff I don’t want to plug into my actual computers (I’ve been playing around with USB based electronics).
I have and 2 network switches and essentially have two separate networks.. an internal and external.
People sometimes give me a hard time about spending this much money on hardware (also a lot of this is rackmount.. I got the rack for free, but I do pay the rackmount tax on the hardware) .. but the way I see it, it’s my hobby.. and dollar for enjoyment, it’s actually not bad. Compared to people who spend the same money to spend 2 weeks on a boat .. I think it’s a good investment.
My point is, if you are trying to save money for something, ok, look for cost effective. If you’ve got the money though.. this is your hobby.. don’t be afraid to spend some cash if you know it’ll make you happy.
For as little as 18 euro pro month I have a VPS with 2 Gb memory and 80 Gb diskstorage and a terabyte networktraffic. At this price you can not have a suitable inhouse testlab.
I have a pretty powerful home system.
Core i5-2500k OC to 5.0GH
16GB RipJaw 1600Mhz DDR3
7TB Hard drive total
1 60 GB SSD Vertex 2 and 1 30 GB Vertex SSD
2 x 5830 XFire
Dual GB Lan
Gigabit Wireless
XFi Titanium Sound card
Blueray Burner and Reader
750Watt 80A Professional Grad Powersupply
This is both my desktop and test system, I just do what I want on it and when I want on it. It runs Gentoo Linux with several cross environments installed for embedded development. I don't see the point on keeping a second computer for tinkering with when you can do all of that on a good desktop computer.
I get old, cheap hardware. Lots of it available. Install Linux on it and do what I want with it. If it breaks I can get replacements for next to nothing.
I'm not exactly the Department of Energy.
You can also find some really neat stuff to work with old hardware in salvage sales - Data Acquisition stuff, cameras, etc.
Don't write off the old stuff, not unless you actually need a super computer for something.
A feeling of having made the same mistake before: Deja Foobar
Off-hand, I'd say a big determining factor is going to be whether or not this "testing" has a lot of do with networking.
If part of what you're doing revolves around configuring routers or switches, or even a lot of tinkering related to how workstations interact with a server or servers, I don't think you want to look at the cloud as a viable option. In my opinion, hosted applications/servers in the cloud only make sense for production systems ready for deployment and regular use (which equates to said configuration providing some sort of cost savings or profit generation), *or* scenarios where multiple people need to collaborate on some sort of software project. A team working on coding an app might find hosting it in the cloud very beneficial, as different individuals sign in, contribute their work/changes to the code-base, and let everyone see and test the results.
When a core part of what you're trying to observe or experiment with has to do with the infrastructure (LAN network) and what's being seen on the client/server side as things are manipulated, I think you'd be better off having all of it on-site to work with it physically. I mean, sure, you could virtualize both a server and a workstation on some cloud-hosted system and test them remotely -- but the LAN network between the two would be completely virtual/simulated since they're both really on the same piece of hardware.....
I've had as many as 9 individual machines at home for testing, development, and support. If I had to do it today, I would:
- Build one honking machine to host servers and network resources. 8-16GB RAM, 4 or more cores, Maybe an SSD to help perk things up.
- Build another honking machine for emulating the various desktop-type stuff you'll want to do. 8GB RAM might work here, but why scrimp?
Choosing the VM environments is the hard part. Virtualbox and Xen are obvious choices, though if you're in Windows all the time you can use machine #2 with Virtual PC. The server machine would benefit from true VM, not just a hypervisor, I think, but you'll get a hundred responses praising one and damning the others.
And of course a GbE switch. I might add in a fairly simple machine to do admin with, and for DR. This machine might store images, depending on how things work. You will be doing lots of images. My simple VPC disk images run up to 12+GB real quick, and I keep 6-7 around all the time, with a dozen more I drop in for special projects. This is at work, where things ahve come full circle - I once again have more storage at home than I do at work, and my Internet is faster. Oh, and I can barely send anyone any data due to security, which is not entirely bad.
You may become enamored of a SAN. This should be a fourth machine if you spin your own. Don't virtualize it.
deleting the extra space after periods so i can stay relevant, yeah.
I purchase older generation off lease equipment off of ebay for use in my own home lab.
I currently have around 4 2u servers with dual dualcore or quadcore cpu's. About the only thing you need to purchase are hard drives. For that I picked up 1 3u 15 bay drive chassis with dual amd dualcore cpus, 16G ram and running about 8 500G drives and 8 1TB drives. It has 4 gig network adapters that I use lacp with for link aggregation on a cheap managed switch that supports lacp.
The only problem is my switch, I paid around $60 for a dell 24port gig switch, but the dell switch kind of sucks, I should have spent a little more (okay a lot) and picked up a cisco.
I came, I conquered, I coredumped
It basically depends on what you are trying to do.
I have to test software on a variety of platforms. This includes Win2k to Win7, OS X 10.5-10.7 and various *nix platforms so I have a ton of VMs, something like 40 I think.
I've got a Dell Precision 490 with dual xeons, 32GB ram and 3 TB of space. I picked it up without the hard drives off of Craigslist for $500.
I am running Win7 with VMware workstation, but I am about to switch to ESXi so that I lose the Win7 overhead.
It does most of what I need, but I have to have a couple of MacBooks due to OS X licensing. I suppose I am technically out of compliance because I moved my Lion VM from my MBP to the Dell, but it works better for what I need to do.
I use a poweredge 2900 with 24gb of ram and 10tb. Loaded with VMware ESXi (free). It can handle multiple servers and workstations running at the same time. Initial cost was high, but a better solution for us long term versus cloud or renting, etc. credit card payments were lower than other solutions and I own it. there are trial editions of just about every server OS out there. So other than the hardware, its been free to tinker otherwise.
Let's make like a bird... and get the flock outta here.
There's really no point in having a mashup of hardware collecting dust in the basement anymore. Linux KVM (kernel based virtualization) is free and quite stable. Other options abound by Vmware and Oracle too if you like to click EULAs.
Not quite sure why anyone would want to go the hardware route anymore unless they are developing for specific architectures that are not supported by the hypervisor.
Join the Slashcott! Feb 10 thru Feb 17!
That is very clever. How long have you been huffing?
Hugs and kisses,
Juan Epstein
When I learned SharePoint, I used a dedicated low-end PC with Linux and VMware Server. I installed a second hard drive to dedicate to the virtual machines. It took some time to boot the environments, but it worked. If I were to do it today, I'd go with a better desktop, load up on memory and use multiple hard drives in whatever RAID configuration made sense. This can scale out to multiple desktops, NAS, managed switches, etc. I'd probably use VMware Hypervisor instead of Server as well. Boot it off a USB flash drive or SD card if you want. Consider power, cooling, noise, and space in your hardware selection.
I have around 30 virtual machines running on a single tower server running ESXi. Solaris/x86, Windows XP, 7, server. A dozen different Linux installations. (Mostly used for software development, with a Jenkins-based continuous integration system building code across different platforms, spinning VMs up as needed).
Pretty much anything I could do with a rack of servers, I can do remotely with a bunch of VMs. I can access the console remotely, reboot, power-on, power-off virtual machines remotely. I can create a new VM and install an OS on it remotely. Add network switches, replumb the network between them. Mount or eject ISO images.
And there's stuff you can't do easily with physical servers that you can with VMs. Take a system snapshot, change something or test something, then roll back to the snapshot.
For "production" use there are a lot of tradeoffs between hardware and virtualization, but to play with or develop on it's hard to beat.
I have 8 cores, 16 gigs of RAM and a bit under 3 terabytes of disk. It cost a lot less, burns less power and makes *much* less noise than the rack of servers it replaced. You could get by with a lot less than that if you limited the number of VMs you had running concurrently.
You may want to re-look at the clouds, for around $500 a year you can really have a nice lab, spin up and down systems till your heart is content. The big issue is you need to be aware of your usage. Power off systems your not playing with. It is a play ground, don't leave them running just because. Virtual Box is a great idea if you want to buy a new box. but you will spend more in two years on that PC, 4core minimum and 16GB of ram, (my minimum) and power, then you will renting a little slice of a cloud. It's computing in a new way, and need to be looked at differently. Plus you have easy access to it from anywhere.
CloudI is an example of a BSD licensed cloud (PaaS type). To read more about it: http://cloudi.org/faq.html#1_Clouds .
I decided on a portable lab myself. I got tired of being stuck in one place while tinkering, testing and learning various software and operating systems.
I bought an ASUS republic of gamers mode # g74sx-bbk7 from BestBuy for my lab setup. It has 17.3" screen, laptop iCore 7, 8GB of Ram (Expandable to 16) Nvidia GTX 560m 2GB? , HDMI, DVDRW drive, built in cooling system. backlit keyboard. 4G Wimax embedded.
I installed VMWare Workstation. I Run Server 2008 R2 with two Windows 7 Clients in VMWare and it works great also have virtualized Server 2003, Windows XP, Ubuntu. I can make imagse in the virtual envrionment. I can have the 2008 server act as a domain server with the Windows XP and 7 virtual workstations as clients. I have also setup several ubuntu test servers as web servers. I find the 8gb of ram is plenty. I also use this same computer to run my Traktor S4 with Traktor Pro 2 Software. I havent found anything it cant do yet. Best of all I can take it to the coffee house, my backyard whereever I want and not have to worry about wires, a spaghetti or a mess of network equipment and power cords.
If you dont want to spend the money on VMWare Workstation you could use VirtualBox but I will say it does do some strange things at times and it is not 100% stable with Windows 8 developer preview.
You could also experiment with virtual appliances as well if you got VMWare though
If you want to run a web server to run a website this probably would not be a solution for you.
It actually is on clearance at best buy right now for 1049.99 they started at 1199.99 if you want a desktop unit they also have a beefy Asus Desktop for 799.99
Get three small cat3k L2/L3 capable switches from say eBay. You'll be able to do most LAN topologies with those at both layer2 and layer3.
Hey, at least it wasn't goatse.
You can pick up not-so-dead socket 775 procs, ddr2, boards, used power supplies from recycling centers, I've always been able to get this crap for free from restaurants doing jobs for em', I find it, and offer to haul it away. They think the stuff is broken, but I've seen that they usually have one issue (Dead psu, hard drive for some reason causing a short, improperly seated memory from Carlos trying to copulate with it.) You get 6 of these old PoS's, strip em' apart and you get 3-4 good ones. Word of the wise: Chain restaurants usually waste the most money on having new hardware (Where Software changes occur once every 2-3 years), so you can usually get the best hardware from them.
Throw in some entry-level gaming video cards (Which you'll need to buy), and you've got yourself a nice little farm to run everything on, in a much sexier way than vm-ing it up.
You could probably cut your losses by selling the com port PCI cards that are likely to be in the PoS machines, unless you really need a null modem network.
I have two broken display Thinkpads with duocore cpu's. You can get these for cheap on craigslist or perhaps free from your friendly IT department. Laptops are ideal because they have greatly reduced power consumption compared to desktops, and have built-in battery backup. Set them up with your preferred linux distro and either eucalyptus or openstack, and bam! you have your own private cloud. Eucalyptus can run EC2 machine images if that's what you're lab testing. Plus, even with typical 5 year old laptops, you'll have more compute power than four basic EC2 instances.
... since VMware Player version 3.1.3 you can create VMs using the free VMware Player.
Get a custom build and put in a few extra hard drives to play with. That's it.
I normally buy used rackable systems (from somewhere like unixsurplus.com). For $400 you can get 4 cores ( a couple years old) and 8GB of ram. That should be enough to run a small lab at very little cost.
I do security
After a long time using standard PCs in the home for development I've finally splashed out on a HP DL160 G6.
I've done this because I'm fed up with replacing power supplies, fans and running out of motherboard memory capacity. In my experience the HP rackmount servers (almost) never break down and you can stuff serious amounts of memory into them (the DL160 G6 has 18 SIMM sockets). My server spec is 2 x quad core cpu + 4 x 3.5 inch disks + 40GB RAM. Paid about GBP 1000 for the server (second user) off EBAY then added 32GB RAM. Its a good deal if you compare it with a standard size motherboard which can take that sort of memory and a pair of CPUs and you add in the cost of a good case and power supply.
With a good server you can concentrate on virtulisation and your testing and be not forever repairing things. Quality always pays off in the long term.
Andy
Right now I am seeing quad-core Xeon 1U Dell rackmounts with 146 GB hard drives and 2GB or more of RAM in corporate dumpsters. Lots of desktop stuff too.
Hardware is free as long as you can afford to spend your labor & time on it.
VirtualBox is nice if you don't plan on doing anything intense as it is not as robust as VMWare.
I have also heard about some driver issues, but can't remember the details.
"That's right...I said it."
I have an AMD Phenom II with 6 cores and 16 GB of RAM. I believe the CPU, motherboard and memory cost me like 400$. You can grab latest release of VMware Workstation which supports running ESX/ESXi as VMs. You can grab EMC Celerra VSA or FreeNAS for iSCSI solution to present some disks to the ESX VM.
I would agree with this, as I have found some hardware that was good enough for my purposes where it may not have been good enough for the service provider. I have also found decent hardware at the local electronics recycling center for the county. It is amazing what some people throw away. Of course there is a lot of junk also.
You can run it on 32bit cheap x86 hardware and have "containers" to run whatever you want on it. I run most of my personal lab stuff of of it without the investment of an expensive piece of hardware. The containers can be any Linux distro. If you need Kernel control there's a KVM element of it if you have hardware that supports VT.
http://robertbchase.blogspot.com/2010/09/proxmox-32bit-hypervisor.html
A word of wisdom. Don't fall into the buying lots of hardware trap like I did. Before long you will end up with a full rack of equipment in your basement with multiple SAN's and networks supporting lots of crazy stuff. It's a huge time hole in itself.
http://www.systemv.org/site.html
Virtual box (non ose) and spend your money on physical hardware and play with iSCSI. If you had some spare dough, buy some Cisco switches to play with also. This will keep you busy for a while.
Much the same as echoed here
Buy a nice processor, more cores the better, speed per core isn't such a huge issue, mainly making sure things stay repsonsive and nothing saturates the cores is what you are after (more cores means less chance of that happening). An AMD Phenom X6 would be ideal. Getting a dual CPU board puts prices much higher and is probably not worth it.
Buy as much RAM as you can. 16GB is nice, more would probably not be wasted. Most desktop motherboards only have 4 DIMM slots and so you will likely be limited to 4 x 4GB chips. 8GB are great but cost a small fortune. Check crucial.com to see what a board can support.
The hard disks would be best RAID'd up in some fashion. It probably is worth spending the money on a PERC5/6 RAID card from Dell. This can take normal SATA disks. get 3 or 4 cheap 1TB disks at £35 each and RAID01 them or RAID5 them. Or, have two RAID 1 arrays and split VMs and potential SAN data. If SSD prices continue to fall sharply, get a 240GB one and on it's own with thin provisioning it should be scary fast and not wobble under load.
I think a 1 port gigabyte network card would suffice but if you want to play with switches then get a 4 port one. Gb NICs and switches would be nice but really, 100Mb is likely fine in a test lab (unless you do VMotion externally). A virtual machine running something like Vyatta can do routing for you very well and is free. You only really miss physically cabling things (which is useful) and cisco commands. For concept learning it is more than fine.
Software wise I would go with ESXi5. Works on basic SATA drives (unlike ESXi4) if you don't want a RAID card and is the leader by a mile at the moment. Installing it also give 60 days free trial of all the licenceed features. Install this to an SD card, boot from it and create the data stores. In 60 days simply reinstall to the SD card, mount the data stores again and carry on as normal. A bit of a pain but doable. The free version is fine also one it expires, has features that others don't like 'as needed' memory allocation and thin provisioning which saves massive amounts of disk space. You can also do virtual ESXi hosts inside esxi to test things as you need to in Virtual Center (vmotion, emotion HA etc) without hosing your 'real' box every two months.
So, a basic box of a 4 core machine, 16GB RAM, two RAID1 arrays and ESXi5 gives amazing potential. Create a SAN only datastore on one array, use the other for VMs. You can install ESXi inside itself, migrate machines using the SAN VM you setup earlier (open filer etc..). You can create multiple subnets and route traffic using Vyatta to simulate an enterprise network with many sites. You can create fairly complex windows environments and configurations. You can test link failures by powering off Vyatta and see what happens to your Exchange 2010 DAG groups etc
Really, with such a small outlay (£500-£600 really) you can do almost anything. Such a powerful and flexible setup even 5 years ago would have cost much more. Buy a microsoft technet licence and there is literally nothing you can't test or play around with. Even if the machine isn't being used and you have an interest in other computing hobbies (i.e. 3d rendering) you can create a 'do-it-all' vm that you can remotely connect to and use as a machine to install trial software, run render nodes, convert your flags to MP3's etc It really will be very useful. Just as long as you had another PC to use to connect to it.
If you can wait, some Intel Z68 motherboards will support 8 DIMM slots, giving even more RAM potential. RAM will be the first thing to run out, so it might be worth waiting to see how that goes. Or, try one of these boards, takes cosumer RAM and can handle 2 opteron 6 core CPUs. Not a bad place to start!
http://www.hotukdeals.com/deals/asus-kcma-d8-atx-socket-c32-motherboard-260-delivered-ballicom-co-uk-1030548
Hi, Maybe I do not have the same needs as you but I create my own lab at home with some mini itx motherboard and small factor cases and I am very happy with them. I use Xen as server and a lot of vms. I use a mini itx/small box for my pfsense router/firewall (and wireless access point) and a small 8 ports vlan aware switch from hp. I can give here more detailed information if you want. Also, it depends on what you want to test at home (clustering, cloud, ha, security, web, etc)
I use a combination of physical, virtual and cloud infrastructure to meet my needs. As a Windows developer / consultant I use:
- Budget Asus Laptop with a Pentium Dual core CPU, upgraded to 8GB of memory and a 120GB SSD with wireless keyboard/mouse and 24" LCD monitor
- Azure hosted TFS in the cloud
- Hosted DNN CRM portal for my website and Email, Blogger for my Blog
-- Drop box for file sharing, Mozy for offsite backup
- Homebuilt i7-2600K with 16GB or memory and 4 300GB Raptor HDD's in RAID 1+0 running Hyper-V for my VM's
--- VM's include Domain Controllers, Build Controller/Agents and integration test machines which I can snapshot and restore at will
I've spent around $1500 for all of my hardware and around $100/mo for all of my various online services and registration fees including code signing certs and 30/5 internet service. This gives me everything I need to code / build / deploy /test / repeat in a very small and affordable package.
NewEgg Wishlist I had sitting around...
$429 (1) ASUS KGPE-D16 SSI EEB 3.61 Server Motherboard Dual Socket G34 AMD SR5690 DDR3 800/1066/1333 ...
$184 (1) CORSAIR Professional Series Gold AX850 (CMPSU-850AX) 850W ATX12V v2.31 / EPS12V v2.92 80 PLUS GOLD Certified Modular Active
$499 (2) AMD Opteron 6128 Magny-Cours 2.0GHz Socket G34 115W 8-Core Server Processor OS6128WKT8EGOWOF
$775 (8) Kingston 8GB 240-Pin DDR3 SDRAM DDR3 1333 ECC Registered Server Memory Model KVR1333D3D4R9S/8G
$798 (8) Western Digital RE4 WD5003ABYX 500GB 7200 RPM SATA 3.0Gb/s 3.5" Internal Hard Drive -Bare Drive
Add a server chassis that can hold the SSI EEB board form factor... I think the memory is compatible...
So for around $2600 you can build one hell of a server... Plus that board can add a 'PIKE' board for SAS drives and SATA 3.0 even...
S1200KP seems to have ECC support in mini itx form factor. Any downsides to that one besides having only two memory slots?
My home "test machine" is a Xeon E1220, which ironically is the least expensive (at retail) i7 system by around $80. I stuck 16GB of (admittedly expensive fully buffered) RAM in a Supermicro motherboard and added an IBM ServerRAID M1015 (8 port SAS card that supports 3TB drives, they sell on Ebay for $75 - $100). It all sits in a nice 3U chassis that I've had for years. Most of the hardware in the machine is devoted to running one little FreeBSD VM that supports ZFS for all the drives I have in that machine, but since the RAM/CPU needs of that machine are so small, relatively speaking, the same box is home to a couple full-time Windows Servers and a Centos VM and I'm still only using half the CPU and RAM resources available to that system.
The interesting thing about that setup is that it's really not as ridiculous as it sounds: $200 for the CPU, $150 for the motherboard, $200 worth of RAM.
The thing I use at work for my virtual machine needs uses a couple quad core Xeon 5000-series CPUs and has 8GB RAM. It's an Intel-branded server that I picked up off Craigslist for $250. I use it to run four Server 2003 guests and it's more than adequate for the use it has been given. It's probably a little too loud to leave running in a home (2U servers generally are), but I suspect someone with sufficient motivation could find a workaround for that.
-- I wanna decide who lives and who dies - Crow T. Robot, MST3K
Just reflash the bios, if ever you get a HP then...
aaaaaaa
Why do so many slashdotters suggest proprietary software? I had a co-worker in the same situation tell me about his plans for a vmware server. I replied by having him come to my desk and typing two lines to start a vm in KVM on debian:
sudo apt-get install qemu-kvm
sudo kvm -m 1024 --cdrom
Install virt-manager to get a GUI to do the same thing.
I am amazed at how easy it is to use free software, yet what sits at the tip of everyone's tongue is proprietary.
This is something the original post didn't really address. However it's critical. The two situations for this I see are as follows:
1) Uptime is minimal, only when Playing...then get the cheapest hardware that you can, old whatever.
2) Uptime is Always. I don't see a list of computing power needed. But a laptop with dual core, maxed out on ram, with Vmware will have about the lowest Amp draw around. Of course you'll be limited to a handful of VM's up at a time, but if you only need 1 or 2 up ALL the time, the the rest are once and a while...this might work OK for you. Also, your host OS could be used for something like a webserver that your not playing with and breaking all the time, but want running all the time.
c) Need lots of computing power which none off the above address...sorry
(no time for spelling or grammar.....sorry) (oh and 1,2,c it's a joke laugh)
Those who can, do.
We recently moved our servers to rackspace and so far that has worked out really well. You have to do a bit of research and figure out what works for you, but for us this was awesome. We can push from local dev machines up to the server and run tests there. It won't work for everyone, but those it does work for, it removes a lot of hassle in dealing with hardware. ~Ben
Utilize GNS3 for your networking labs. Learning how GNS3 and Dynamips (the brains behind GNS3) works will make it a resource viable option. The newest version (0.8.1) brings VirtualBox support as well.
I've had nothing but problems getting network hardware to work under ESXi. Three different NICs and none would work. It's very particular about what it will and will not support in my experience. If you're going to run ESXi, use Intel hardware all the way (chipset, CPU, NICs).
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
Virtualization is the thing. I was fortunate to be able to do this early on (5+ years ago) and I learned a few things along the way:
1) Memory is the thing. VMware and the other hypervisors are really good at making the most out of memory (ballooning, shared memory, de-dupe, etc.), but RAM is cheap now. My setup has 16GB and I can do just about anything I want with this.
2) Disk is even more the thing. My setup is a cluster, but even if it wasn't I'd still use some sort of external disk solution. I have two in my environment. For my "dev and cheap" stuff I use a cheapo windpc+opensolaris/indiana ZFS based NAS box. You can build yourself something that rivals enterprise class stuff for dirt cheap. My other solution is a discard from one of my customer. I have an (older) netapp array. It's clustered and does all sorts of fun stuff. At one point it was an enterprise class solution, so for my lab it's more than enough. You use external storage because your VM's then become transportable. The server you use today, might not be the same tomorrow. Moving the bits around from one server to another is a pain. Having them on external storage makes it really easy: fire up the new server, connect it to the storage, shut down the VM's on the old server, start them up again on the new server, retire the old server. And if you have compatible architectures, you might not even need to do the shutdown the VM's at all.
3) Get your servers from the company you work for (or a friend works for). My lab is Dell 2950 based. I was able to get the servers for next to nothing from a company that had depreciated them and was willing to sell them for nothing. They are not the most current, bestest, fastest thing, but guess what - the time when CPU and memory are the bottlenecks for most stuff is over. Even three year old hardware can run most current OSs more than adequately for a home lab. Granted, if you are going to playing games in your "lab", then this isn't going to work. But then again, it would turn this whole discussion moot anyways.
-- thoughts on one of those things: http://amuyu.com/
I'm having a hard time seeing some of this. Ether it's "spend big $$$" or "go dumpster diving".
While I can appreciate the dumpster diving (because it encourages recycling), the spend big $$$ bothers me, especially the "get teh bleeding edge!!!1!!" comments for Llano. Why spend that when I can get AM3 socket-based devices for cheaper? I just priced a system at Newegg for $520 in new parts, that would get you:
I don't give a rat's ass that AM3+ is the new hotness, I only care that the existing AM3 product line will get their pricing hammered as the newer products come out, which means...I'll be able to incrementally upgrade for cheap in 6-18 months.
Just two months ago I bought a dual xeon 5680 1u server with 48 gigs of ram and dual 147 gig sas drives, dual nics and dual power supplies (850w) for $1600. I spent another $500 transferring it to a tower as I didn't realize how noisy a 1u setup is (it was in on the first floor of my house and I could hear it upstairs). Now I'm about to sell the old 1u server with motherboard to hopefully recoup a portion of the $500 spent for converting it to a tower. So for a little over $2000 I have a computer that is more powerful than several of those where I work.
What do I use such a power house for? Mostly experimenting with new technologies while running all of my websites at home (it'st still probably way too overpowered for what I do, but the speed is nice)
I run a Dell PE2900 and run Proxmox VE on it. Total cost around $1500 so far and entire lab fits in a closet.
Largely on the kind of testing you are going to be doing. For functional software testing, one big box hosting VMs is great. I do it with ESXi, a quad core, 8gb, and several disks. good enough for a virtualized server and a couple of workstations. If that's your target, then more spindles is better. 4 250gb hard drives give better performance than a 1TB drive, because there is no contention for disk access (if you set it up right).
-- "Never underestimate the power of human stupidity." - R.A.H.
Another vote for VMWare ESXi. I acquired a scrapped server-class machine (dual proc, 8 disk RAID, dual NIC, redundant PS), and run all my instances on it. I have a spare machine loaded and powered down as disaster recovery. It's a little loud (lots of fans) but I can't even hear it -- it's in the garage, close to the router. I either remote to it or use the VMWare console from my home office.
The advantage, in my opinion, of using server class machines, even if they're old and slower by today's standards, is that they're more like the hardware (and issues of same) that you're likely to encounter in a job. The built-in redundancy is nice, also. And the fact that you can acquire a few hanger queens fairly easily from which to scavenge parts for spares. You get experience (which you are looking for) and you get reliability (redundancy) and it's profoundly cheap. What's not to love?
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
A HP microserver with ESXi makes a great test server - I've had a number of test servers over the last 10 years and ever so pleased how versatile, cheap and stable this combo is.
My current lab consists of an old HP DC7700 with a Core 2 Duo CPU running SVR 2K8 Enterprise, 6GBRAM 5TB HDD, PERC6 card, (2) 1GBNICS, I use this as my ISO repository, File Server, and Domain controller, I have a Quad Core Core 2 Duo in another white box PC with 6GBRAM, 3TBHDD, and (3) 1GB NICS running Xenserver. My Exchange is virtualized here, and I'm labbing out Xendesktop currently. My 3Rd is an older Xeon server with a 200GB LTO3 tape backup (2) 1GB NICS, Svr 2K8 standard and Symantec Backup Exec. I use this as a backup server. This combination of enterprise and home software and hardware works well. I can lab out almost anything, including iSCSI NAS. I get ~80 to 110 MB/s across the network, which is surprising considering many of my NICS cost 10.00$. My partnerships with Microsoft, Symantec, and Citrix go a long way. I'm able to lab out complicated scenarios before planning client infrastructure. Good luck. Just remember to buy solid workstations which are capable of virtualization and have at least 4 memory slots, and go Gigabit all the way. It's inexpensive these days. You can choose VMWare or Citrix for virtualization, it depends on your preference and requirements.
A lot of my testing is OS "scenario" based, so I stick with virtualization on VMware. Here's my setup:
2x Shuttle SH55J2 - core i3, 16GB RAM each, USB boot
2x Intel PRO 1000 MT adapters (Native NIC support in VMware is very picky)
QNAP TS-410 with 4x 1TB HDD
Asus desktop gigabit switch
All up, approx $1600 AUD. I went as cheaply as I could, maximum bang for buck while keeping it all small, quiet, and low(ish) power draw. It all sits in an AV cabinet in my living room.
I run VMware vSphere accross this using the QNAP as shared storage (NFS), i run my "home network" VM's (6) within this as well as whatever VM's I happen to be playing with. As usual, RAM is the biggest problem when it comes to running VM's, and the Shuttles are already maxed out at 16GB each, so you'll need to be frugal depending on what you're doing. I don't have a requirement for CPU horsepower so the i3's hum along very nicely. A faster QNAP would be nice though, disk access is a little slow on this model.
I use an older model second hand HP server I picked up. You can get one of these bad boys for a few hundred bucks, or if you are lucky you can pick one up for free when you upgrade a small business. 16gb of ram and dual intel xeon's, i run vmware ESXi and use a fat client to interact with it, works a treat
You can get a decent generic barebone from Tigerdirect for less than $300 (have to watch for a deal) with a quad-core processor, 8GB of RAM and a TB hard drive. I have one with Xenserver free version because I like the tools and driver support. I have used VMWare 2 GSX and ESX, then ESX3, VMWare Server free version and ESXi, but have been using Xenserver free version in both test and production for the last three years, though I understand that VMWare's solutions are also very workable. A UPS is helpful as well.
My current test Xenserver has at time of this writing 4 VMs on it - two Linux boxes, a Windows server 2003 and an XP instance, all used for testing and development. I have a Windows 7 instance as well, but it happens to be turned off at the moment. I use an external USB storing snapshots of test VMs - get a clean config, store a snapshot of it, then you can test, muck it up, blow it away then start up a copy of the snapshot without having to re-install. Mine has been running continuously since early summer.
This setup can get you started with minimum cost and effort if you are doing development and functional testing that does not include anything too exotic like clustering or a database with a large transaction volume. You're not going to break any speed records but you can build VMs in all the OS types you want to test and limit the number currently running to 4 or 5, and you'll do just fine.
The size of the fans in a 1U server makes them noiser just as a matter of physics.
Go with a 2U, and you should be good.
I just switched my virtualization environment to Windows 8 Dev Preview.... The new hyper-v stack included bests Server 2008 r2 for local dev work (allows use of wireless network connections for virtual switches)... after years of VMware usage, i can say that hyper-v has finally passed workstation (and esx with server 8)... life isn't that complicated! I would recommend a laptop with i7 or better and max out the ram. All you need besides your personal machine is an external hard drive (usb3 would be nice) for the vhd images. I simply converted my machine to a vhd and now run as a vm (dev env 1)... Then keep a separate vm for each project of significance.
I have an ASUS P6X58D-E with an i7-920 and 24GB running ESXi 5.0. It's perfect for playing around with different OSes and testing software.
Run over to www.vm-help.com for everything you need to know about cheap whitebox virtualization with ESXi. They maintain a HCL and Forums for everything VMWare.
The new version of ESXi supports a LOT more "whitebox" hardware than the older versions - they're clearly responding to market demand for cheaper servers by providing support for common consumer-grade hardware.
Those are illegal and can burn your house down...what is that you say? Test? Not meth?...Oh well that is very different. Nevermind.
Oracle Virtual Box is the way to go in my opinion.
VMWare ESX or VMWare ESXi does not account for a low cost solution. By their nature you are building a test environment that will house 10-20-50 or even a hundred servers/virtualized workstations.
This represents the best way to learn Virtual Center and Virtual Backups and other virtual infrastructure components such as VDI and Site Recovery Manager and HA/FT/NLB/MSCS.
The cheapest way to learn all of that is buy an MSDN subscription, download VMWare ESXi, download virtual center trial (gives you 90 days before reinstall is required). Get a domain name with an email address forwarder to your personal mail otherwise you will never be able to try the commercial products that make virtualcenter and vmware esx work in a real environment. Plan on spending 2500 to 3000 for a course in VMWare ESX in order to get the VCP test (a vmware authorized course is required in order to sit in on the test—look it up). Plan on purchasing vmware licenses one /maybe two for access to the 128 gig of ram on the server and to get FT/HA capabilities.
Next plan on buying a commercial license for 1200 bucks for nexentastor (so you can do things like inter san replication and ISCSI. Nexenta has a free license but it does not do SAN high availability you need to purchase the commercial version for that so you might opt out the learning of SAN to SAN replication and site recovery capabilities.
Next check white list find a compatible motherboard that takes 2-4 processors with up to 8 cores per processor. It’s an imperative since the slow point in a test network is usually not the processors but instead being able to have 64 gig to 128 gig of memory is the top priority.
When you combine that server with power supply that is compatible according to manufacturer of the motherboard and compatible ECC memory notice ECC memory is really a big help for stability and error detection. Look at the white list and make sure that the controller for the raid is compatible. If not plan on spending 500 to 1000 for a raid controller. Next purchase ((6) hard drives 2 TB drives for two RAID Z -1 configurations with 3 drives in each raid), and 2 ½ TB drives for the ESX system installation. You will raid this setup (this is found in nexentastor. Make sure that PCI pass through is supported by your server because you are going to pass the RAID z disks to both sets to two separate virtual guests. One guest being Nexentastor-1 server and the other Nexentastor 2 Server these will serve as your ISCSI units.
Install Nexenta in 2 separate VMs and setup two to 4 virtual networks (Nic cards except to get internet access with this setup is not necessary since you are virtualizing your whole environment. ) plan on allocating 12 gig of ram to each Nexentastor instance (that’s why 128 gig of ram is important). Note that super fast ram is not necessary 800 mhz ram will do fine for this test setup although in a real environment you are adding a second physical server (possible up to 5 more servers with high speed ram) you will still be able to get the full spectrum feel for how this works with the one ESX server with PCI passthrough. That’s 24 gig of ram just used by the virtualized SAN. IT’s definitely a necessity. Remember each guest takes 2-4 gig of ram in a test environment especially if you are going to script fake loads onto the servers to emulate a real environment. So 128-24 leaves you with 104 gig of ram or ability to run 16 VM’s with 4 gig of ram for each or 32 VM’s with 2 gigs of ram each. Remember Server 2008 R2 eats ram so does SQL and exchange.
You will build at least 2 virtual networks but 5-10 is possible. VMKernel requires a separate virtual network to run if it is to support advanced features such as HA and FT. Real networking can get expensive for multiple network cards and bonding and 10 Gig Ethernet, Nexus switches and the like but all this knowledge makes you data center technician capable with 50-80 an hour salary so this isn’t just for fun.
Having a machine with
VMWare ESX or VMWare ESXi does not account for a low cost solution. By their nature you are building a test environment that will house 10-20-50 or even a hundred servers/virtualized workstations.
This represents the best way to learn Virtual Center and Virtual Backups and other virtual infrastructure components such as VDI and Site Recovery Manager and HA/FT/NLB/MSCS.
The cheapest way to learn all of that is buy an MSDN subscription, download VMWare ESXi, download virtual center trial (gives you 90 days before reinstall is required). Get a domain name with an email address forwarder to your personal mail otherwise you will never be able to try the commercial products that make virtualcenter and vmware esx work in a real environment. Plan on spending 2500 to 3000 for a course in VMWare ESX in order to get the VCP test (a vmware authorized course is required in order to sit in on the test—look it up). Plan on purchasing vmware licenses one /maybe two for access to the 128 gig of ram on the server and to get FT/HA capabilities.
Next plan on buying a commercial license for 1200 bucks for nexentastor (so you can do things like inter san replication and ISCSI. Nexenta has a free license but it does not do SAN high availability you need to purchase the commercial version for that so you might opt out the learning of SAN to SAN replication and site recovery capabilities.
Next check white list find a compatible motherboard that takes 2-4 processors with up to 8 cores per processor. It’s an imperative since the slow point in a test network is usually not the processors but instead being able to have 64 gig to 128 gig of memory is the top priority.
When you combine that server with power supply that is compatible according to manufacturer of the motherboard and compatible ECC memory notice ECC memory is really a big help for stability and error detection. Look at the white list and make sure that the controller for the raid is compatible. If not plan on spending 500 to 1000 for a raid controller. Next purchase ((6) hard drives 2 TB drives for two RAID Z -1 configurations with 3 drives in each raid), and 2 ½ TB drives for the ESX system installation. You will raid this setup (this is found in nexentastor. Make sure that PCI pass through is supported by your server because you are going to pass the RAID z disks to both sets to two separate virtual guests. One guest being Nexentastor-1 server and the other Nexentastor 2 Server these will serve as your ISCSI units.
Install Nexenta in 2 separate VMs and setup two to 4 virtual networks (Nic cards except to get internet access with this setup is not necessary since you are virtualizing your whole environment. ) plan on allocating 12 gig of ram to each Nexentastor instance (that’s why 128 gig of ram is important). Note that super fast ram is not necessary 800 mhz ram will do fine for this test setup although in a real environment you are adding a second physical server (possible up to 5 more servers with high speed ram) you will still be able to get the full spectrum feel for how this works with the one ESX server with PCI passthrough. That’s 24 gig of ram just used by the virtualized SAN. IT’s definitely a necessity. Remember each guest takes 2-4 gig of ram in a test environment especially if you are going to script fake loads onto the servers to emulate a real environment. So 128-24 leaves you with 104 gig of ram or ability to run 16 VM’s with 4 gig of ram for each or 32 VM’s with 2 gigs of ram each. Remember Server 2008 R2 eats ram so does SQL and exchange.
You will build at least 2 virtual networks but 5-10 is possible. VMKernel requires a separate virtual network to run if it is to support advanced features such as HA and FT. Real networking can get expensive for multiple network cards and bonding and 10 Gig Ethernet, Nexus switches and the like but all this knowledge makes you data center technician capable with 50-80 an hour salary so this isn’t just for fun.
Having a machine with