Slashdot Mirror
Ask Slashdot: Little Boxes Around the Edge of the Data Center?
First time accepted submitter spaceyhackerlady writes "We're looking at some new development, and a big question mark is the little boxes around the edge of the data center — the NTP servers, the monitoring boxes, the stuff that supports and interfaces with the Big Iron that does the real work. The last time I visited a hosting farm I saw shelves of Mac Minis, but that was five years ago. What do people like now for their little support boxes?"
I make them with ticky tack.
Give me Classic Slashdot or give me death!
put them in VMs!
I don't work in a data center. But I think you might want to look at an HP Proliant MicroServer.
Basically it is an AMD laptop chipset on a tiny motherboard in a cunningly designed compact enclosure. The SATA drives go into carriers that are easily swapped (but not hot-swappable). It's quiet and power-efficient. It supports ECC memory (max 8GB) and supports virtualization.
http://h10010.www1.hp.com/wwpc/us/en/sm/WF06b/15351-15351-4237916-4237918-4237917-4248009-5153252-5153253.html?dnr=1
Silent PC Review did a complete review of an older model (with a 1.3 GHz Turion instead of 1.5 GHz).
http://www.silentpcreview.com/HP_Proliant_MicroServer
SRP is $350, but Newegg has it for $320 (limit 5 per customer).
http://www.newegg.com/Product/Product.aspx?Item=N82E16859107052
Newegg also has 8GB of ECC RAM for about $55, so you can get one of these and max its RAM for under $400.
I just got one and haven't had time to really wring it out, but I did do the RAM upgrade. Despite the tiny enclosure, it wasn't too painful to work on it, and I was impressed by the design. The Turion dual-core processor has a passive heat sink on it, and the single large fan on the back pulls air through to cool everything. (There is also a tiny high-speed fan on the power supply.)
I'm going to use this as my personal mail server. It's cheap enough and small enough that I plan to have at least one put away as a hot spare; if the server dies, I'll power it down, move the hard drives to the spare, and I'll have the mail server back up within 5 minutes. Not bad for a cheap little box.
lf(1): it's like ls(1) but sorts filenames by extension, tersely
Virtualized NTP is about the dumbest thing I've read on /.
Yes, worse than various conspiracy theories and fanboi wars.
No little unsupportable boxes here.
---- Booth was a patriot ----
Last generation's compute nodes. We keep some around for utility functions after decommissioning a large cluster.
Go get a GPS satellite receiver/time server. Actually, get two. Don't screw with time.
THEN, virtualize the rest of the stuff. Monitoring, syslogging, management, patchers, etc.
We've virtualized everything except for
- a Windows DC so that it stays up if the vmware datastores or SAN eats itself in a horrible way.
- The NIS server we have to use on our UX environment due to an ancient regulation. I'm not willing to put up HP-UX VMs for this right now, otherwise it'd be safe in a VM as well.
- Anything we can't virtualize due to licensing/contract/support issues. So our VOIP environments, phone call recording, access control systems for the doors,
My datacenter is getting a lot nicer to look at, and a lot easier to upgrade. I can shift servers or volumes all over the room so I can do live maintenance during the day.
My mom says I'm cool.
Those support tasks don't exactly push hardware to its limit, and most of those tasks are the kind of thing that demands a bunch of redundant servers anyway.
Throw a bunch of "last generation" hardware at the task -- stuff from the "asset reclamation" pile. Leave a few more around as spares. Less disposal paperwork. Works just fine. By the time your last spare fails, you'll have a new generation of obsolete hardware.
--
To be fair, if someone cares enough about time accuracy to understand why that's a dumb idea, they should probably be using a GPS receiver instead of a PC.
NTP server is all about consistency. If it's running in a VM and can be delayed at the whim of the host, do you think it's going to be a very good source of time?
I think its apalling that we do that. Its a horribly expensive way to work in hardware but we do it because we can't be stuffed to deal with operating systems. Most likely a single box and OS instance could do it for you if it was set up correctly.
http://michaelsmith.id.au
If you can't run it on your iPad, it's probably not worth running.
--Management.
rewriting history since 2109
I personally hate and despise people who put non-rackmount kit in racks...
We use various devices.. mostly all 1ru servers of various configs... for eg there are a couple of mini-itx 1ru servers we have that have e350 based mini-itx boards (i really love the e350/e450 boards)... not quite as cheap as the hp n40 microserver, but at least its a rack format.
Then we have a few that run virtualisation here and there for some tasks using kvm (some of those too have e350's in them as the e350's do have the virt'n extensions unlike the intel atoms)... we also have a few that run intel based i3/i5/i7 mini-itx boards... they're quite nice when you need some extra grunt...
some others are based on super micro boards as well though (which are quite cheap and run core i3/i5/i7 cpus rather the xeons). Then some others are old 1ru xeons we no longer need for server tasks...
I can't imagine trying to perform network management with a few mac minis so I'm assuming you're referring to a very small facility? Our new data center was built on 10-gig infrastructure and our NM is appropriately scaled--NetScout Infinistreams connected to Gigamon matrix switches. While the Gigamons were quite expensive they allowed us to utilize fewer Infinistreams while also providing some very cool functionality.
It look a long time for our upper management (those with the dollars) to come around to the notion that, in order to realize the full investment made in the data center, true network management needed to be baked in from the start.
That's true. My company uses IBM BladeCenter servers bundled into a VM cluster. The bang-for-buck at the Time were the 4-core Opterons... That easily scaled to 4-cPUS for 16-cores.. (That could probably be higher now). The beauty of AMD. Moving into this space is that the blades could be swappable with the current hardware.
But rather than rowed of boxes, VM is the better way to go.
We are using a couple Soekris boxes for some basic monitoring. They are lightweight atom processors with no active cooling and it's designed with networking in min. 4 Gig-E ports on the 6501, and you can get up to 8 more thanks to 2 PCI-E slots available in the rackmount version. Since we are using an mSATA SSD on the board we have no moving parts, so nothing mechanic to fail.
I don't know everything.
NTP servers are NOT about consistency, they are about making badly designed protocols, such as NFS, capable of limping, instead of just falling on their face.
If the requests on these protocols used a client timestamp for the client's idea of the current time, then the server on receiving the request could look at its idea of the current time, and arrive at a delta before it actually did anything other than enqueue the request locally.
Then when the server responded with a non-"now" timestamp in any client response, it could apply this delta to the response value, and as far as the client was concerned, it and the server would have synchronized ideas of "now", without resorting to all of this NTP BS or worrying about clock drift, or anything.
I lobbied very strongly to try to get this fixed in NFSv4; maybe we will get our collective heads out of our butts by NFSv5.
Why even have a dedicated server for NTP? It's not as if it's the bad old days of Win NT and one service per box due to memory leaks. If you've got special hardware for an external time source that can be hooked up to an existing server, because the actual software to hand out time consumes buggerall resources. It consumes so little that redundancy is a matter of just configuring whatever machines you've already got to be as many NTP servers as you want just at a lower stratum than whatever you really trust. They'll keep time reasonably well for a fairly long time while the custom time source is off.
...I don't want it in my datacenter. If you have no budget for non-revenue-generating boxes for services like DNS, NTP, etc. then upgrade the server hardware you tore out of production after the last upgrade cycle with SSDs and low-wattage processors & put it back into service for your internal needs.
Otherwise get a few Dell R210s or some other small cheap rack server with an IPMI 2.0 BMC and get on with your business. Any money saved by buying "mini-PCs" (or whatever you want to call them) for any datacenter computing hardware you plan to rely upon at all will be burned the first time you have to drive to the datacenter and physically babysit some cheap machine because it didn't have IPMI.
Answer: VMware VMs.
"Flyin' in just a sweet place,
Never been known to fail..."
If you care enough to use a GPS receiver instead of a network time source, you should also care enough to get the antenna on to the roof... We have many such time sources controlling timing in the basements of buildings, but the antenna always ends up on the mast.
I don't operate a datacenter, but for virtualized servers in an office, I always enable the NTP server functionality in the hypervisor, have it sync to a stratum-1 time source, then advertise that address via DHCP and DHCPv6 for my guests and workstations (and visiting cell phones) to use. Being the definitive time source, I also tell the hypervisor to automatically set the clock on the guests, then give a virtualized AD domain controller (if any) the PDC FSMO role to set the Windows domain time. I have sites with two or three hypervisors running NTP, and it seems to work well. Not sure if it will scale to your environment, OP, but it may be worth mulling over.
I wont dare tell you how much you pay per KWh but 38 watts at 9 cents per KWh would cost me just shy of $30 per year: http://www.citytrf.net/costs_calculator.htm
Virtualisation is great, but there are a few things that cause horrible chicken/egg problems if you virtualise them.
So I'd reserve at least two separate boxes to "do infrastructure". DNS, NTP, remote logging, trap receiving, bastion, and so on. You simply plunk a unix on them and put the individual services in jails or the local equivalent. Don't even need much in the way of performance, so any old 1U box will do fine. Heck, a soekris or an alix board will do. Those are short enough that you can stick'em in any old wiring closet too. Great for geographically dispersing.
If you're stumping up for infrastructure that can host hundreds of VMs, then of course that is enough capacity to also run "little boxes", but it'd be stupid to not also shell out the little extra to make your infrastructure robust, instead of risking hypervisor dependencies on not-yet booted VMs in your private cloud, or whatever you'd call it. "Seems to work" is not enough: Turn off the entire datacentre and then try and cold boot it, remotely. If it's fully virtualised including necessary basic supports, it'll take more time and trouble than if you don't virtualise the pillars on which you built up the rest.
If all I had was exactly two boxes, I'd still run NTP and local DNS next to the hypervisor, not under a guest. NTP in particular; I've had my fill of (windows) boxes claiming to be stratum two yet being off by two minutes because they only update once a week. Of course, on a virtualised unix it'll be much less, but I don't want to find out the hard way the VM distorted the timekeeping in unexpected ways later, so this is one thing that needs its own box. There are similar scenarios for the other basics, but I'll leave them as an exercise. The gains of virtualising, saving a bit on hardware and power, simply do not outweigh the trouble when you can least afford it.