Making Best Use of Data Center Space: Density Vs. Isolation

jfruh writes The ability to cram multiple virtual servers on a single physical computer is tempting — so tempting that many shops overlook the downsides of having so many important systems subject to a single point of physical failure. But how can you isolate your servers physically but still take up less room? Matthew Mobrea takes a look at the options, including new server platforms that offer what he calls "dense isolation."

Comments

Re:Blade Servers aren't "new server platforms"

[Anonymous Coward]

Heck, 13 years ago at a Canadian federal government job we swapped our web servers for blades.

Which was pretty bleeding-edge at the time, since the first blade server was 2001. So not sure what your point about the government is - they weren't late to the party, far from it.

Re:Twins

[K. S. Kyosuke]

Judging from their figures, "extremely dense" ("375 GFLOPS/kW") is actually fairly mediocre, considering that the Blue Gene/Q figures with POWER chips is several times better. That's obviously not exactly relevant for generic server workloads, but I wonder what the figures on those are.

this article

[Idimmu Xul]

is not good.

Simple

[ledow]

Put all your eggs in one basket.
Then make sure you have copies of that basket.

If you're really worried, put half the eggs in one basket and half in another.

We need an article for this?

Hyper-V High Availability Cluster. It's right there in Windows Server. Other OS's have similar capabilities.

Virtualise everything (there are a lot more advantages than mere consolidation - you have to LOVE the boot-time on a VM server as it doesn't have to mess about in the BIOS or spin up the disks from their BIOS-compatible modes, etc.), then make sure you replicate that to your failover sites / hardware.

Re:Simple

[ledow]

I have just put in a Blade / VM configuration at a school (don't ask what they were running before, you don't want to know).

Our DR plan is that we have an identical rack at another location with blades / storage / VM's / etc. on hot-standby

Our DDR (double-disaster recovery!) plan is to restore the VM's we have to somewhere else, e.g. cloud provider, if something prevents us operating on that plan.

The worries I have are that storage is integrated into the blade server (a SPOF on its own, but at least we have multiple blade servers mirroring that data), and that we are relying on a single network to join them.

The DDR plan is literally there for "we can't get on site" scenarios, and involves spinning up copies of instances on an entirely separate network, including external numbering. It's not a big deal for us, we are merely a tiny school, but if even we're thinking of that and seeing those SPOF's, you'd think someone writing their article into Slashdot would see that too.

All the hardware in the world is useless if that fibre going into the IT office breaks, or a "single" RAID card falls over (or the RAID even degrades, affecting performance). It seems pretty obvious. Two of everything, minimum. And thus two ways to get to everything, minimum.

If you can't count two or more of everything, then you can't (in theory) safely smash one of anything and continue. Whether that's a blade server, power cord, network switch, wall socket, building generator, or whatever, it's the same. And it's blindingly obvious why that is.

Physicalisation

[Pegasus]

Look at the likes of HP Moonshot and AMD Seamicro. Those are some nice toys to play with ...

Mod parent up.

[khasim]

Nor are they "isolated". All of the blades connect to the same backplane.

And moving VM's between individual blades is a hassle unless you use some form of shared storage. Which makes them even less "isolated" but more redundant.

This reads more like he just wanted to show off that he calls blade servers "dense isolation".

So is it better to have a bunch of isolated servers which reduces the VM domino effect in exchange for increased hardware maintenance? Or just a few massive servers and be ready for the 4 am call to replace a CPU at any given moment?

VM is not magic. Also look into "fail over".

If you have to be called in to replace a CPU at 4 am then you have not planned correctly.

Re: TL;DR

[saloomy]

He should consider using virtualization to increase his uptime since he is worried about multiple important systems on a single server. Virtualization gives you such good yields in consolidation, you can come out ahead while still using redundancy features like VMware FaultTolerance. Your vm runs "in-step" on two hosts, and will survive even if either host fails. Just requires 2X the used memory. That's still only the most extreme case though like for databases, as most servers should be able to survive a reboot (which is what happens when your host dies and there is capacity left in your cluster. The VM powers back up on another host.

Blade servers blow

[swb]

I'll accept the idea that somewhere somebody has so many servers and so little space that a blade center was the only way they could achieve the density they needed.

Except I've never seen it -- all the blade centers I've ever seen have been partially full and the equivilent 1U and 2U servers probably would have fit in the same or less space than the blade chasis was occupying.

And almost always there's a mongolian clusterfuck when they decide to add blades to the chasis -- which they inevitably do, because they have so much money sunk into the blades that there's no way out from under it.

The mongolian clusterfuck is the result of the byzantine cofiguration rules each vendor has for determining a blade's NIC or FC mapping with the blade center's (overpriced) internal switch bays. Half or full height? LoM or mezzanine slot? Which mez slot? Which blade slot? Oh, you want an extra NIC on that blade? Sorry, the mapping requires an additional switching module which will cost you more than any decent L3 48 port gig switch.

Whatever the savings from the blade center (and maybe in some metered situation there is power savings of couple hundred watts) is easily lost in hours of troubleshooting when trying to do something different.

Blade centers always look like some kind of pre-virtualization version of server consolidation that became obsolete once 24U of servers could easily be run on 8U or less of VM host and SAN. They would be a lot more interesting if their mapping regeimes weren't hard wired -- blade advocates give me blahblah "point of failure" about a switchable/configurable backplane.

Re:Blade servers blow

[Jaime2]

HP Blades put a 2U server in 1.6U or a 1U server in 0.8U. The only downside is that there is very little room for local storage. If you are virtualizing, SAN storage is inevitable anyways. The power backplane is just a hunk of copper, and all the intelligent stuff is duplicated, so there isn't really a single point of failure - but I wouldn't go blades unless I was at the scale of needing at least three blade chassis so it would be possible to shut one down and not interrupt production. The most legitimate complaint about blades is that they're too dense. A rack that consumes 56 kilowatts is a challenge to cool.

I've found that two fiber modules - at $6K each - are cheaper than two fiber cards in each of 16 individual servers. Don't buy the embedded switches, use the ones that act as a rear mounted ethernet port, along with your switches of choice. If you do want switching on the chassis, get the ones with 10G uplinks so you can drastically reduce the amount of wiring you need.

As for configuration complexity - don't use blades if you are a shop where every server is different. Their ideal use case is one where you need 100 or more of the same configuration - like a VM farm.

Re:Blades

[mikey1134]

The SAN is usually less of a single point of failure because they usually have a lot of redundancy built-in, redundant storage processors, multiple backplanes, etc. You're right that off-site replication is still important, but usually more for whole site loss than storage loss.