Slashdot Mirror
Ask Slashdot: Computer Test Lab Set-Up For Home?
An anonymous reader writes "For as long as I've been playing around with computers I've had a home test lab. I found it to be a great learning tool. However, I haven't invested much money into it lately and because of aging hardware I can't get what I want out of it anymore. So a revamp is in order. I've looked into several cloud vendors for a box I can rent to do some virtualization, but it doesn't seem to be cost effective or practical. What are your thoughts on it? What set-up do you have at home for tinkering? Have you looked into hosted solutions for this?"
Buy a computer.. Put >8GB of ram in it (i would recommend 2GB per VM, and 2GB for the host). Maybe some nice fast disks..
Load VMWare ESXi, or another OS and virtual machine software of your choice..
The ability to snapshot and restore things will save you so much time testing things, you'll wonder how you used to get things done before. Maybe, setup a second system or laptop for things like wireless testing, drivers, etc.. things you can't simulate in a VM.. but with the virtual networks in most VM's, you can setup some very, very complex networks...
What are we going to do tonight Brain?
It is not entirely clear what you want to tinker with. What do you want to test? Are you wanting to tinker with hardware? Using different software? Writing software? If the latter, what kind? To what end? This is a useless summary of your question.
Would imagine it depends very heavily on what you are actually doing
I’ve got:
- A powerful desktop,
- Large fairly expensive file server and a cheap backup file server (same capacity, but cheap hardware and drives)
- Several old boxes (mostly previous desktops and stuff I rescued from people who were going to throw them out) one of which is acting as a virtual machine host..
- Two intel atom based boxes. One I use for a whole bunch of random stuff (for instance, all the various UPSen I have are plugged into it, and it coordinates a shutdown of everything when any of them run low on battery. The other I use as my hardware tinkering box.. when playing with stuff I don’t want to plug into my actual computers (I’ve been playing around with USB based electronics).
I have and 2 network switches and essentially have two separate networks.. an internal and external.
People sometimes give me a hard time about spending this much money on hardware (also a lot of this is rackmount.. I got the rack for free, but I do pay the rackmount tax on the hardware) .. but the way I see it, it’s my hobby.. and dollar for enjoyment, it’s actually not bad. Compared to people who spend the same money to spend 2 weeks on a boat .. I think it’s a good investment.
My point is, if you are trying to save money for something, ok, look for cost effective. If you’ve got the money though.. this is your hobby.. don’t be afraid to spend some cash if you know it’ll make you happy.
For as little as 18 euro pro month I have a VPS with 2 Gb memory and 80 Gb diskstorage and a terabyte networktraffic. At this price you can not have a suitable inhouse testlab.
I get old, cheap hardware. Lots of it available. Install Linux on it and do what I want with it. If it breaks I can get replacements for next to nothing.
I'm not exactly the Department of Energy.
You can also find some really neat stuff to work with old hardware in salvage sales - Data Acquisition stuff, cameras, etc.
Don't write off the old stuff, not unless you actually need a super computer for something.
A feeling of having made the same mistake before: Deja Foobar
Off-hand, I'd say a big determining factor is going to be whether or not this "testing" has a lot of do with networking.
If part of what you're doing revolves around configuring routers or switches, or even a lot of tinkering related to how workstations interact with a server or servers, I don't think you want to look at the cloud as a viable option. In my opinion, hosted applications/servers in the cloud only make sense for production systems ready for deployment and regular use (which equates to said configuration providing some sort of cost savings or profit generation), *or* scenarios where multiple people need to collaborate on some sort of software project. A team working on coding an app might find hosting it in the cloud very beneficial, as different individuals sign in, contribute their work/changes to the code-base, and let everyone see and test the results.
When a core part of what you're trying to observe or experiment with has to do with the infrastructure (LAN network) and what's being seen on the client/server side as things are manipulated, I think you'd be better off having all of it on-site to work with it physically. I mean, sure, you could virtualize both a server and a workstation on some cloud-hosted system and test them remotely -- but the LAN network between the two would be completely virtual/simulated since they're both really on the same piece of hardware.....
I've had as many as 9 individual machines at home for testing, development, and support. If I had to do it today, I would:
- Build one honking machine to host servers and network resources. 8-16GB RAM, 4 or more cores, Maybe an SSD to help perk things up.
- Build another honking machine for emulating the various desktop-type stuff you'll want to do. 8GB RAM might work here, but why scrimp?
Choosing the VM environments is the hard part. Virtualbox and Xen are obvious choices, though if you're in Windows all the time you can use machine #2 with Virtual PC. The server machine would benefit from true VM, not just a hypervisor, I think, but you'll get a hundred responses praising one and damning the others.
And of course a GbE switch. I might add in a fairly simple machine to do admin with, and for DR. This machine might store images, depending on how things work. You will be doing lots of images. My simple VPC disk images run up to 12+GB real quick, and I keep 6-7 around all the time, with a dozen more I drop in for special projects. This is at work, where things ahve come full circle - I once again have more storage at home than I do at work, and my Internet is faster. Oh, and I can barely send anyone any data due to security, which is not entirely bad.
You may become enamored of a SAN. This should be a fourth machine if you spin your own. Don't virtualize it.
deleting the extra space after periods so i can stay relevant, yeah.
I purchase older generation off lease equipment off of ebay for use in my own home lab.
I currently have around 4 2u servers with dual dualcore or quadcore cpu's. About the only thing you need to purchase are hard drives. For that I picked up 1 3u 15 bay drive chassis with dual amd dualcore cpus, 16G ram and running about 8 500G drives and 8 1TB drives. It has 4 gig network adapters that I use lacp with for link aggregation on a cheap managed switch that supports lacp.
The only problem is my switch, I paid around $60 for a dell 24port gig switch, but the dell switch kind of sucks, I should have spent a little more (okay a lot) and picked up a cisco.
I came, I conquered, I coredumped
I use a poweredge 2900 with 24gb of ram and 10tb. Loaded with VMware ESXi (free). It can handle multiple servers and workstations running at the same time. Initial cost was high, but a better solution for us long term versus cloud or renting, etc. credit card payments were lower than other solutions and I own it. there are trial editions of just about every server OS out there. So other than the hardware, its been free to tinker otherwise.
Let's make like a bird... and get the flock outta here.
There's really no point in having a mashup of hardware collecting dust in the basement anymore. Linux KVM (kernel based virtualization) is free and quite stable. Other options abound by Vmware and Oracle too if you like to click EULAs.
Not quite sure why anyone would want to go the hardware route anymore unless they are developing for specific architectures that are not supported by the hypervisor.
Join the Slashcott! Feb 10 thru Feb 17!
When I learned SharePoint, I used a dedicated low-end PC with Linux and VMware Server. I installed a second hard drive to dedicate to the virtual machines. It took some time to boot the environments, but it worked. If I were to do it today, I'd go with a better desktop, load up on memory and use multiple hard drives in whatever RAID configuration made sense. This can scale out to multiple desktops, NAS, managed switches, etc. I'd probably use VMware Hypervisor instead of Server as well. Boot it off a USB flash drive or SD card if you want. Consider power, cooling, noise, and space in your hardware selection.
Sounds like I could trade your system for a nice car.
If he is trying to save cash I do not think your solution is cost effective. However, you can get systems now for cheap with 8 gigs of ram and at least a hex core that could run software development fine. I do admit simulating server loads wont work but that is why you try to get the boss to pay for it then :-)
http://saveie6.com/
I have around 30 virtual machines running on a single tower server running ESXi. Solaris/x86, Windows XP, 7, server. A dozen different Linux installations. (Mostly used for software development, with a Jenkins-based continuous integration system building code across different platforms, spinning VMs up as needed).
Pretty much anything I could do with a rack of servers, I can do remotely with a bunch of VMs. I can access the console remotely, reboot, power-on, power-off virtual machines remotely. I can create a new VM and install an OS on it remotely. Add network switches, replumb the network between them. Mount or eject ISO images.
And there's stuff you can't do easily with physical servers that you can with VMs. Take a system snapshot, change something or test something, then roll back to the snapshot.
For "production" use there are a lot of tradeoffs between hardware and virtualization, but to play with or develop on it's hard to beat.
I have 8 cores, 16 gigs of RAM and a bit under 3 terabytes of disk. It cost a lot less, burns less power and makes *much* less noise than the rack of servers it replaced. You could get by with a lot less than that if you limited the number of VMs you had running concurrently.
You may want to re-look at the clouds, for around $500 a year you can really have a nice lab, spin up and down systems till your heart is content. The big issue is you need to be aware of your usage. Power off systems your not playing with. It is a play ground, don't leave them running just because. Virtual Box is a great idea if you want to buy a new box. but you will spend more in two years on that PC, 4core minimum and 16GB of ram, (my minimum) and power, then you will renting a little slice of a cloud. It's computing in a new way, and need to be looked at differently. Plus you have easy access to it from anywhere.
Get three small cat3k L2/L3 capable switches from say eBay. You'll be able to do most LAN topologies with those at both layer2 and layer3.
Hey, at least it wasn't goatse.
Oh I agree cost does sound to be something to think of, but he did ask what we have for setups :-), On a completely pointless rating my system is a 7.6 on Windows Experience.
You can pick up not-so-dead socket 775 procs, ddr2, boards, used power supplies from recycling centers, I've always been able to get this crap for free from restaurants doing jobs for em', I find it, and offer to haul it away. They think the stuff is broken, but I've seen that they usually have one issue (Dead psu, hard drive for some reason causing a short, improperly seated memory from Carlos trying to copulate with it.) You get 6 of these old PoS's, strip em' apart and you get 3-4 good ones. Word of the wise: Chain restaurants usually waste the most money on having new hardware (Where Software changes occur once every 2-3 years), so you can usually get the best hardware from them.
Throw in some entry-level gaming video cards (Which you'll need to buy), and you've got yourself a nice little farm to run everything on, in a much sexier way than vm-ing it up.
You could probably cut your losses by selling the com port PCI cards that are likely to be in the PoS machines, unless you really need a null modem network.
Wrong, the Xeon is entirely meant for server side computing which is not relevant to desktop computing. Comparing the i5 and i7 series to the Xeon's is like comparing a 18 wheeler to a sports car. Both are meant for different tasks.
... since VMware Player version 3.1.3 you can create VMs using the free VMware Player.
I normally buy used rackable systems (from somewhere like unixsurplus.com). For $400 you can get 4 cores ( a couple years old) and 8GB of ram. That should be enough to run a small lab at very little cost.
I do security
After a long time using standard PCs in the home for development I've finally splashed out on a HP DL160 G6.
I've done this because I'm fed up with replacing power supplies, fans and running out of motherboard memory capacity. In my experience the HP rackmount servers (almost) never break down and you can stuff serious amounts of memory into them (the DL160 G6 has 18 SIMM sockets). My server spec is 2 x quad core cpu + 4 x 3.5 inch disks + 40GB RAM. Paid about GBP 1000 for the server (second user) off EBAY then added 32GB RAM. Its a good deal if you compare it with a standard size motherboard which can take that sort of memory and a pair of CPUs and you add in the cost of a good case and power supply.
With a good server you can concentrate on virtulisation and your testing and be not forever repairing things. Quality always pays off in the long term.
Andy
Right now I am seeing quad-core Xeon 1U Dell rackmounts with 146 GB hard drives and 2GB or more of RAM in corporate dumpsters. Lots of desktop stuff too.
Hardware is free as long as you can afford to spend your labor & time on it.
VirtualBox is nice if you don't plan on doing anything intense as it is not as robust as VMWare.
I have also heard about some driver issues, but can't remember the details.
"That's right...I said it."
I would agree with this, as I have found some hardware that was good enough for my purposes where it may not have been good enough for the service provider. I have also found decent hardware at the local electronics recycling center for the county. It is amazing what some people throw away. Of course there is a lot of junk also.
Virtual box (non ose) and spend your money on physical hardware and play with iSCSI. If you had some spare dough, buy some Cisco switches to play with also. This will keep you busy for a while.
Hi, Maybe I do not have the same needs as you but I create my own lab at home with some mini itx motherboard and small factor cases and I am very happy with them. I use Xen as server and a lot of vms. I use a mini itx/small box for my pfsense router/firewall (and wireless access point) and a small 8 ports vlan aware switch from hp. I can give here more detailed information if you want. Also, it depends on what you want to test at home (clustering, cloud, ha, security, web, etc)
My home "test machine" is a Xeon E1220, which ironically is the least expensive (at retail) i7 system by around $80. I stuck 16GB of (admittedly expensive fully buffered) RAM in a Supermicro motherboard and added an IBM ServerRAID M1015 (8 port SAS card that supports 3TB drives, they sell on Ebay for $75 - $100). It all sits in a nice 3U chassis that I've had for years. Most of the hardware in the machine is devoted to running one little FreeBSD VM that supports ZFS for all the drives I have in that machine, but since the RAM/CPU needs of that machine are so small, relatively speaking, the same box is home to a couple full-time Windows Servers and a Centos VM and I'm still only using half the CPU and RAM resources available to that system.
The interesting thing about that setup is that it's really not as ridiculous as it sounds: $200 for the CPU, $150 for the motherboard, $200 worth of RAM.
The thing I use at work for my virtual machine needs uses a couple quad core Xeon 5000-series CPUs and has 8GB RAM. It's an Intel-branded server that I picked up off Craigslist for $250. I use it to run four Server 2003 guests and it's more than adequate for the use it has been given. It's probably a little too loud to leave running in a home (2U servers generally are), but I suspect someone with sufficient motivation could find a workaround for that.
-- I wanna decide who lives and who dies - Crow T. Robot, MST3K
Just reflash the bios, if ever you get a HP then...
aaaaaaa
Why do so many slashdotters suggest proprietary software? I had a co-worker in the same situation tell me about his plans for a vmware server. I replied by having him come to my desk and typing two lines to start a vm in KVM on debian:
sudo apt-get install qemu-kvm
sudo kvm -m 1024 --cdrom
Install virt-manager to get a GUI to do the same thing.
I am amazed at how easy it is to use free software, yet what sits at the tip of everyone's tongue is proprietary.
This is something the original post didn't really address. However it's critical. The two situations for this I see are as follows:
1) Uptime is minimal, only when Playing...then get the cheapest hardware that you can, old whatever.
2) Uptime is Always. I don't see a list of computing power needed. But a laptop with dual core, maxed out on ram, with Vmware will have about the lowest Amp draw around. Of course you'll be limited to a handful of VM's up at a time, but if you only need 1 or 2 up ALL the time, the the rest are once and a while...this might work OK for you. Also, your host OS could be used for something like a webserver that your not playing with and breaking all the time, but want running all the time.
c) Need lots of computing power which none off the above address...sorry
(no time for spelling or grammar.....sorry) (oh and 1,2,c it's a joke laugh)
Those who can, do.
I've had nothing but problems getting network hardware to work under ESXi. Three different NICs and none would work. It's very particular about what it will and will not support in my experience. If you're going to run ESXi, use Intel hardware all the way (chipset, CPU, NICs).
If I mod you up, it doesn't necessarily mean I agree with what you've said, sorry.
Virtualization is the thing. I was fortunate to be able to do this early on (5+ years ago) and I learned a few things along the way:
1) Memory is the thing. VMware and the other hypervisors are really good at making the most out of memory (ballooning, shared memory, de-dupe, etc.), but RAM is cheap now. My setup has 16GB and I can do just about anything I want with this.
2) Disk is even more the thing. My setup is a cluster, but even if it wasn't I'd still use some sort of external disk solution. I have two in my environment. For my "dev and cheap" stuff I use a cheapo windpc+opensolaris/indiana ZFS based NAS box. You can build yourself something that rivals enterprise class stuff for dirt cheap. My other solution is a discard from one of my customer. I have an (older) netapp array. It's clustered and does all sorts of fun stuff. At one point it was an enterprise class solution, so for my lab it's more than enough. You use external storage because your VM's then become transportable. The server you use today, might not be the same tomorrow. Moving the bits around from one server to another is a pain. Having them on external storage makes it really easy: fire up the new server, connect it to the storage, shut down the VM's on the old server, start them up again on the new server, retire the old server. And if you have compatible architectures, you might not even need to do the shutdown the VM's at all.
3) Get your servers from the company you work for (or a friend works for). My lab is Dell 2950 based. I was able to get the servers for next to nothing from a company that had depreciated them and was willing to sell them for nothing. They are not the most current, bestest, fastest thing, but guess what - the time when CPU and memory are the bottlenecks for most stuff is over. Even three year old hardware can run most current OSs more than adequately for a home lab. Granted, if you are going to playing games in your "lab", then this isn't going to work. But then again, it would turn this whole discussion moot anyways.
-- thoughts on one of those things: http://amuyu.com/
I run a Dell PE2900 and run Proxmox VE on it. Total cost around $1500 so far and entire lab fits in a closet.
Largely on the kind of testing you are going to be doing. For functional software testing, one big box hosting VMs is great. I do it with ESXi, a quad core, 8gb, and several disks. good enough for a virtualized server and a couple of workstations. If that's your target, then more spindles is better. 4 250gb hard drives give better performance than a 1TB drive, because there is no contention for disk access (if you set it up right).
-- "Never underestimate the power of human stupidity." - R.A.H.
Another vote for VMWare ESXi. I acquired a scrapped server-class machine (dual proc, 8 disk RAID, dual NIC, redundant PS), and run all my instances on it. I have a spare machine loaded and powered down as disaster recovery. It's a little loud (lots of fans) but I can't even hear it -- it's in the garage, close to the router. I either remote to it or use the VMWare console from my home office.
The advantage, in my opinion, of using server class machines, even if they're old and slower by today's standards, is that they're more like the hardware (and issues of same) that you're likely to encounter in a job. The built-in redundancy is nice, also. And the fact that you can acquire a few hanger queens fairly easily from which to scavenge parts for spares. You get experience (which you are looking for) and you get reliability (redundancy) and it's profoundly cheap. What's not to love?
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
My current lab consists of an old HP DC7700 with a Core 2 Duo CPU running SVR 2K8 Enterprise, 6GBRAM 5TB HDD, PERC6 card, (2) 1GBNICS, I use this as my ISO repository, File Server, and Domain controller, I have a Quad Core Core 2 Duo in another white box PC with 6GBRAM, 3TBHDD, and (3) 1GB NICS running Xenserver. My Exchange is virtualized here, and I'm labbing out Xendesktop currently. My 3Rd is an older Xeon server with a 200GB LTO3 tape backup (2) 1GB NICS, Svr 2K8 standard and Symantec Backup Exec. I use this as a backup server. This combination of enterprise and home software and hardware works well. I can lab out almost anything, including iSCSI NAS. I get ~80 to 110 MB/s across the network, which is surprising considering many of my NICS cost 10.00$. My partnerships with Microsoft, Symantec, and Citrix go a long way. I'm able to lab out complicated scenarios before planning client infrastructure. Good luck. Just remember to buy solid workstations which are capable of virtualization and have at least 4 memory slots, and go Gigabit all the way. It's inexpensive these days. You can choose VMWare or Citrix for virtualization, it depends on your preference and requirements.
You can get a decent generic barebone from Tigerdirect for less than $300 (have to watch for a deal) with a quad-core processor, 8GB of RAM and a TB hard drive. I have one with Xenserver free version because I like the tools and driver support. I have used VMWare 2 GSX and ESX, then ESX3, VMWare Server free version and ESXi, but have been using Xenserver free version in both test and production for the last three years, though I understand that VMWare's solutions are also very workable. A UPS is helpful as well.
My current test Xenserver has at time of this writing 4 VMs on it - two Linux boxes, a Windows server 2003 and an XP instance, all used for testing and development. I have a Windows 7 instance as well, but it happens to be turned off at the moment. I use an external USB storing snapshots of test VMs - get a clean config, store a snapshot of it, then you can test, muck it up, blow it away then start up a copy of the snapshot without having to re-install. Mine has been running continuously since early summer.
This setup can get you started with minimum cost and effort if you are doing development and functional testing that does not include anything too exotic like clustering or a database with a large transaction volume. You're not going to break any speed records but you can build VMs in all the OS types you want to test and limit the number currently running to 4 or 5, and you'll do just fine.
I just switched my virtualization environment to Windows 8 Dev Preview.... The new hyper-v stack included bests Server 2008 r2 for local dev work (allows use of wireless network connections for virtual switches)... after years of VMware usage, i can say that hyper-v has finally passed workstation (and esx with server 8)... life isn't that complicated! I would recommend a laptop with i7 or better and max out the ram. All you need besides your personal machine is an external hard drive (usb3 would be nice) for the vhd images. I simply converted my machine to a vhd and now run as a vm (dev env 1)... Then keep a separate vm for each project of significance.
I have an ASUS P6X58D-E with an i7-920 and 24GB running ESXi 5.0. It's perfect for playing around with different OSes and testing software.
Run over to www.vm-help.com for everything you need to know about cheap whitebox virtualization with ESXi. They maintain a HCL and Forums for everything VMWare.
The new version of ESXi supports a LOT more "whitebox" hardware than the older versions - they're clearly responding to market demand for cheaper servers by providing support for common consumer-grade hardware.
ECC Memory doesn't make a test machine, There is absolutely no benefit to buying expensive memory that can handle error coding for a machine to play around with. Your professional system is either a joke or waste of a system. Being you seem to think you need ECC memory shows you have either no experience or no practical knowledge and if you work in the field please for the love of god don't release a product.
Those are illegal and can burn your house down...what is that you say? Test? Not meth?...Oh well that is very different. Nevermind.
maybe with some mods but rackmounted servers for the most part give no consideration to sound levels. Not that they should since most racks are in data centers. I am sure that there are a few rack cases that are quite for use in the music industry.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
I don't understand the question. ESXi is a bare metal virtualization server from VMWare. KVM in it's usual meaning is a Keyboard/Video/Mouse switch. You usually use a KVM if you have several machines to which you need access. ESXi is a free (as in beer) way to virtualize several machines onto one server. This is not only usually cheaper, but it's also easier to manage.
So I guess to answer the question as asked, the difference is that ESXi has multiple instances on a few servers, and using a KVM kinda assumes you have one instance per physical server. (The ESXi console is not very interesting and I've never felt the need to access it from my office. I use the VMWare console application to manage and access the instances.)
VMWare ESX is the Enterprise bare metal solution, and has a bunch of management features that ESXi does not have. But for home you probably don't need it.
If you're practicing for VMWare certification, you can download a 30 day ESX demo and get some practice with the Enterprise features. I've never done that. ESXi works fine for me. At one time I had ten distinct web instance exposed to the outside world, plus spare instances for development, running on one server in my garage, with a second server as disaster recovery. My clients gradually moved on or went out of business, and as I had gone on to a different career I haven't pursued new clients. So the server is currently only serving my blog, daughter's blog, family web page, and a few blogs for friends and family. Currently I'm only running two instances, production and development. But creating others as needed is trivial.
Oliver's law of assumed responsibility: If you're seen fixing it, you will be blamed for breaking it.
Oh I agree cost does sound to be something to think of, but he did ask what we have for setups :-), On a completely pointless rating my system is a 7.6 on Windows Experience.
Wow, your ePeen is without a doubt considerably larger than mine, bravo sir.
..Mullah or Pope, Preacher or Poet, who was it wrote: "Give any one species too much rope and they'll fuck it up"?
On a serious question, how do you quantify that number to mean anything?