New Data Center Standard

mstansberry writes to tell us that the Telecommunications Industry Association (the people who brought you the CAT standards for unshielded twisted pair cabling) recently published a 148 page document meant to standardize the design considerations for every single aspect of a data center. The standard covers everything from site selection to rack mounting methods.

Comments

twisting

[seanadams.com]

Speaking of CAT standards, has anyone else had a good look at the differences between CAT3, CAT5, etc?

CAT5 just seems to be twisted a little tighter, but CAT6 actually modifies the twist gradually, in a cycle that repeats every few feet, with each pair 90 degrees "out of phase" from the next. Plus theres (sometimes) a plastic "spine" in there to maintain spacing and/or bend radius. It's not obvious to me how varying the twists-per-foot along the cable should help - anyone know?

Re:twisting

[Mr. Flibble]

because of the extra twisting there is less crosstalk. The less crosstalk, the more information you can pump down the wire, thus the difference between Cat 3,5 and 6.

(FYI, crosstalk is interference from a parralell channel in the wire)

Re:twisting

[rusty0101]

Not sure entirely myself, however as a thought, having a constant twist (different from one pair to the next, but the same for the length of the cable) could set up a situation where two cables are in phase at several points alog the cable run, and some signal transfer may hapen.

Varying the twist rate along the run of a pair, as well as doing what you can to keep it out of phase with other pairs by braiding, or other means would make it possible to set up a longer cable run without viable phase transfer points that could cause signal bleed between pairs.

However that's just conjecture on my part. I am sure someone will come along who can give us the math to show that my conjecture is entirely wrong.

-Rusty

Re:twisting

[SoloFlyer2]

Varying the twist rate only helps when u have several cables together ( Think 4 or 5 cables in a conduit ). Basically it just reduces the chance that 2 cables next to each other are going to have exactly the same twist rate.

In other words it reduces cross-talk between cables :)

Re:twisting

[william_w_bush]

oh fyi, this doesn't help at lower speeds, but common lvds serdes nowadays hit upwards of 10Gbps, so this kind of thing makes a big difference at that point because the wavelength is small enough that the difference becomes possible to vary in the cabling.

basically for 1gbps its just starting, past 5gbps (10g copper spec calls for this though a working group is trying to run 10g on cat5 (boggle)) you need this kind of thing, and it happens to cut down on rf interference and reception, another handy thing, considering how hard it is to get some of these things to pass fcc (been there, done that).

Re:twisting

[DaEMoN128]

I am BICC certified. There is a difference between CAT3 and CAT 5. The twists are much more pronounced on CAT 5. It may not look like it, but there is a big differnece. There is also different amounts of twists per pair. CAT 6 is something I havent had much hands on with (we use mostly fiber for that stuff). CAT 3 has much lower frequency response per pair than CAT 5. A good cable tester can actually verify that for me if you can get your hands on one (see network analyser). Just from looking online, CAT 6 has a minimum 250MHz bandwith while CAT 5 has a minimum 100 MHz bandwith per pair. http://www.lanshack.com/cat5e-tutorial.asp check here. It says CAT 5e is the same as CAT 6 but CAT 6 is manufactured to a higher standard. I guess that the fab tolerances are tighter for Cat 6.

Re:twisting

[william_w_bush]

also yes and no, the added twisting makes the crosstalk elimination better at different freqencies. at 1ghz a single turn is enough to cause the antenna effect, while at 10mhz (original spec) you need like half a meter or something.

Re:twisting

[lithium100]

I'm no expert but fundamentally it comes down to Maxwell's equations. A current is induced by a magnetic field in a wire by a changing magnetic field - ie; an ac signal of some frequency. Similarly, a changing electrical current in a wire will create a changing magnetic field and hence "crosstalk".

There is one caveat though and that is only the orthogonal components (to the wire) of the magnetic field induce a current and so by twisting the wires you minimize the orthogonal components.

At least that what I remember from fields studies back at uni - has been a while!

Re:twisting

[lithium100]

Take a look at a CAT5 cable. There is practically no insulation at all so the dielectric permitivity would be hardly any better than air. Thats WHY they started twisting it so that heavy insulation wasn't needed as it is in coax.

Re:twisting

[budgenator]

the diameter of the insulater arround the center conductor of a coaxial cable has a lot to do with the capacitance of the cable, such as the center conductor is one plate of the capacitor and the shield is the other. The capacitance is needed to cancel out the inductance of the cable, the product of the cable's capacitance, inductance and resistivity gives a cable a characteristic impeadence such as a 75 ohm cable for your TV.

When all of the impeadances match, the power or energy (can't remember which, and they are different technicaly) is maximised.
One of the radars we used to have used a coax cable to the feed horn, the center conductor was about 1/8 copper pipe surrounded by about a 1 1/2 inches of insulation.

Same is true for twisted pairs, the thickness of the insulation effects the capacitance of the pair and the adjacent pairs, smaller diameter insulation gets you more cross-talk at higher frequencies.

Re:twisting

[ajs318]

When you pull a bell-rope, it stretches slightly. Then the stretched bit shrinks back to how it was and a higher-up bit of the rope is stretched. The stretched bit works its way up the rope to the wheel, which only moves when the last bit of rope snaps back. All this happens far too fast for you to see, but it does happen. {You might be able to see it in a Slinky spring loosely stretched out, especially if you have a camcorder that can do slow motion playback}. Due to the physical properties of the rope and the mechanism at the bell end, there is a small but finite delay between pulling the rope and the bell ringing.

The same thing happens with electrical signals. They do not travel along wires instantaneously. There is an entire subdiscipline of electrical engineering dedicated to the study of transmission lines, but here are some of the points relevant to this discussion.

Every conductor carrying a current radiates a magnetic field. If you could have two wires absolutely coincident and carrying the same current in opposite directions, then the fields they radiate will cancel out exactly and thus not interfere with anything. Obviously you can't get them absolutely coincident, because real matter takes up space. But it turns out that intimate proximity is good enough for real world applications. So, we twist the wires to keep them together ..... this way we don't have to have a huge bulky insulation jacket forcing them together, and can fit more wires into the same space. We can help matters by ensuring that the load on the far end is perfectly resistive: i.e. that every bit of energy put into it actually does work {or just gets turned into heat} rather than getting stored and released.

Now, any wire that is not absolutely straight looks like a coil. Electrons travelling along a coil behave as though they have inertia {due to energy being stored as a magnetic field and then released} and the faster they are moving, the harder it is to persuade them to change direction. If you have a coil with many turns and a socking great lump of steel up the middle, it takes awhile to build up a current from a battery because energy is being stored in magnetising the steel. {Water flow analogy: imagine a turbine in the pipe with a heavy flywheel. It takes an effort to get it up to speed, and it wants to keep turning -- and trying to shove water along -- even after the water stops.} When there are no more molecules to line up with the field, the core is saturated and the coil behaves as a simple resistor until the current changes. Disconnect the battery, and the electrons will actually take some time to come to a halt. That's why you often see a diode -- and maybe a resistor in series with the diode, if it's a really monstrously inductive load like a railway train motor* -- across a relay or motor switched by a transistor. The diode gives the current a path to flow through without having to arc across {a pure current source without a resistor in parallel with itself behaves as an infinite voltage source; most things become good conductors if you apply a high enough voltage}. Air obviously cannot store as much energy in the form of magnetism as steel; but at a high enough frequency, the current will have changed direction before magnetic saturation occurs. Which is why VHF inductances usually have only a few turns on a piddly little ferrite core, and UHF inductances often are literally just bent bits of wire. {Of course, these can even be printed}.

Twisting two wires together actually forms a transformer. It's not a brilliant one, but it will work as one at high frequencies. Each wire induces a current in the other. The induced currents should cancel one another out perfectly. But of course two wires twisted together also form a capacitor. So your twisted pair cable is actually this huge inductive, capacitive, resistive nightmare of a thing.

A cable also has a characteristic impedance. When you put a battery to one end, a pulse current flow

Not this again ...

[Trusty Penfold]

How boring ... who wants to work somewhere identical to the last place. And identical to your friends' places of work.

How about letting a bit of originality in once in a while?

Oh yeah ... and I'm not bloody paying $250 just to make more work for myself.

---
jon_edwards@spanners4us.com

Re:Not this again ...

[drsmithy]

I don't imagine this would be an especially effective means of encouraging compliance either... "we'll tell you what to do for the low, low price of $250!!"

Out of the costs involved in setting up a data centre, $250 wouldn't even qualify as a rounding error.

Re:Not this again ...

[Jekler]

I can't tell if you're being serious. Assuming you're being serious...

Originality and creativity have certain places in the world. Just because you have guidelines and standards doesn't mean you can't be creative. Programming languages have standards, that doesn't mean programmers can't create original programs. If there were no coding conventions and standards, you'd almost never be able to examine someone elses code. "Wait a second, why are all the integer variables stored as strings? And I think this guy uses + signs as assignment operators..."

Standards are there to prevent entropy, not prevent innovation. Every writer, producer, actor etc. has to write a proposal, film a pilot, and go through all the same stages that everyone else has to. Joss Wheddon, Chris Carter, Jon Stewart... they all had to adhere to the same process that their predecessors did, that didn't stop them from being original and succeeding, it prevented their ideas from being ignored because the people reading the proposal didn't need to interpret it, they already knew the format it was going to be in.

Standards make sure that our ideas are understood universally. In the case of a data center, it ensures that we can all store and retrieve data in a unified way. All the people who need that data don't need to figure out some proprietary system. When you contract services from a standards-compliant center, you don't need to hear the line "That might be what you're used to but... this is OUR way of doing things."

Doomed to failure?

[Toby The Economist]

Be interesting to see if it's useful.

When you specify something like a cable, it's straightfoward to get it right, because the job the cable does and the way it's used is very well understood and doesn't vary between users.

With something complex like a data center, there's so much variance in how they're operated, exactly what they do, where they are, etc...having a standard may well *not* fit everyone's needs, either because their needs were not perceived or understood at the time or because their needs simply cannot be met by the standard.

--
Toby

Re:Doomed to failure?

[JanneM]

With something complex like a data center, there's so much variance in how they're operated, exactly what they do, where they are, etc...having a standard may well *not* fit everyone's needs, either because their needs were not perceived or understood at the time or because their needs simply cannot be met by the standard.

True.

But when you have a formal standard, you have something to measure against. Every aspect of the data center design is not only standardized, but the how's, why's and therefore's are spelled out. If you suspect the standard doesn't meet your needs in some respect (a clear lack of surround sound for late-night fps tournaments, say), it makes it clear exactly how your criteria changes the requirements, and it makes it much easier to see how it could impact the rest of the design.

So even if you use not one single recommendation (we need the disco ball, damnit!), you have something reasonable and well documented to compare against, which makes your job easier.

Pity it's $250 for a peek

[darnok]

I'm curious to see what this document contains: whether it's an ITIL-like view of the world (e.g. a data centre runs on change management, capacity management, problem management, ...), a hardware based view (e.g. a data centre needs a raised floor to duct cables, air conditioning, secure access, racks, ...) or something else.

Just not curious enough to pay the price to find out

/. effect

[OneArmedMan]

I wonder if they considered defences to the /. effect when writing this.

Steal the F*ing Manual

[fyoder]

Just not curious enough to pay the price to find out

Seriously. If all manuals were that expensive there would have been no 'RTFM'. It would have been 'STFM'.

Cheap datacentres

[lamasquerade]

I welcome this standard with open arms. I look forward to the not-too-distant day when I will be able to buy 100m(sq) of Standard Data Centre on eBay for $25. No more un-backed-up un-RAIDed hard drives for my mp3s!

Re:Hum?

[Samari711]

[this post intentionally left blank]

PDF HAS BEEN LEAKED CONTENTS FOLLOW

[gutterandthestars]

A data center shall consist of hardware, appropriate HVAC, and gigabytes of pornography.

great news for colocation customers

[pokka]

This is great news for people who host servers in colocation facilities.

If you've ever tried to find a place to host your server in the past, you've probably found that not only does the price wildly fluctuate between hosts for no apparent reason, but also it's very difficult to determine exactly what you're getting, even if you take the time and effort to actually visit the site.

I think that the disorganized fashion of colo services allows people to charge ridiculous prices
and get away with things that they wouldn't be able to do in a more stable competitive environment (like charging ridiculous amounts for bandwidth overage and support).

With some sort of standard in place, vendors will be forced to compete on more even ground, prices will be more reasonable, and users won't be afraid to leave their current colo provider because the next one could potentially be even worse.. Not that it will be perfect, of course - just somewhat better.

Re:Link to the document

[Anonymous Crowhead]

Does anybody have a link to the document yet, since $1.6 per page of bs is a bit too much.

Slashdot moral concept #7: If item is perceived to suck, stealing - oops sorry, forgot Slashdot moral concept #6 - infringing it is allowed.

Example: "If $band would put out better songs, maybe I'd buy their album. Until then, I will continue to use BitTorrent to get their material."

Alternatives to THIS book

[xmundt]

Greetings and Salutations.
          Interestingly enough, a quick google search for "data center design" comes up with more hits than one can shake a stick at, ranging from free to fairly inexpensive (under $100.00). I have to admit that I wonder if THIS magnum opus has anything in it that these OTHER resources do not cover.

          It never ceases to amaze me at the number of books out there that are supposed to be useful learning tools that are nothing more than a slightly changed rehash of the man pages for a given program.

          Regards
          Dave Mundt

Re:Right answer, wrong question

[Anonymous Coward]

To "detune" the coils. If the twists were equidistant (especially in the straighter parts) it would resemble a tuned antenna.

Raised Floors?

[Anti-Trend]

I can't afford the $250 at present, but I wonder if they finally did away with raised floors. It wasn't too bad of idea around 40 years ago, but we've got cool modular racks now that make that concept moot, at least IMHO. Plus raised floors look weird, are fairly expensive to implement (especially for smaller firms with little cash), and get really nasty under there over time. Besides, telco has done without that design for quite a long time, seems to have worked out fine for them.

-AT

Re:Raised Floors?

[DaEMoN128]

After making my living as an installer... raised floors are a GOD SEND!! They are a pain, but they look 1000 times better than cable tray or ladder rack with 2000 cables in them. They make routing cables easier. It is no fun setting up some rig in order to hang cable tray 20 ft. off the ground. You dont want ladder rack the whole way... nothing wrong with it, but why when you can use a cable trough with a cover that allows the use of a pull tape. Raised floors help with cooling (forced air from under the cabnet). Also, they dont have to get nasty under there.... just make sure your standards for presentation are clearly stated in the contracts and enforce them.

Re:Raised Floors?

[Anonymous Crowhead]

I wonder if they finally did away with raised floors.

At my colo, they run cold air from the HVAC under the raised floors and suck it up through the cabinets with a fans (and presure from the HVAC). They cool the cabinets, not the entire facility. It's odd being in a colo that is warm after freezing my ass off in previous facilities, but temperature monitering equipment of ours tells us that the closed cabinets stay quite cool.

Re:Raised Floors?

[Linker3000]

You would have loved the computer room I did some work in during the 1980s - the builders got the dimensions wrong and there was a 4 foot void under the tiles instead of 4 inches!! The owners had to have special pedestals made to hold up the tiles and they actually put some servers under the floor!

Re:Raised Floors?

[wirefarm]

Did they later convert that space to a cubicle farm?
I think I worked there.

Re:Raised Floors?

[marquis-cablewitch]

But without raised floors where am I supposed to hide the bodies?

Re:Raised Floors?

[Chanc_Gorkon]

Raised floors are definitely something you need. What we did was run the backbone cable to patch panels in the server rows. The cabling to the servers are overhead racks with troughs for fiber. This was a huge improvement to the old room. We pretty much bought a switch and prewired every panel. Now when we need to add a server, we have out cisco guy add it to the switch config(we also give him the MAC address at this time too), they tell us which jack to use and we get our wire expert to run the cable from the patch panel to the server. It takes far less time to get new equipment wired becasue we don't run a cable from the equipment to the switch. The only hard part is power, and even that isn't hard. Our Pseries racks all take the same power supply so we had twist locks placed at strategic locations. When we get a new server, if we need another power supply, we order it to and plug it into a empty twist lock receptical. Standardization of Data Centers is sorely needed. The types of standard don't mean every center will look the same, but it will mean that all equipment in the future will have the same or similar requirements for network drops and power. This makes data center planning MUCH easier.

Re:I hope it specifies floor monkeys have unix ski

[Anonymous Crowhead]

My two favs:

#1. Driving all the way to the data center at 4 am to find the keyboard plugged into the mouse port - "The keyboard's not responding", says the "engineer".
#2. Driving all the way at to the data center at 3am to see /sbin/fsck-y on the console - "It says command not found", says the "engineer".

Installed X and AIM after that and made the "engineers" read the commands off that provided they could get that working.

AMEN

[DaEMoN128]

I agree completely. The "industry standards" for many things are used alot, even if they dont partularly apply to your site. I would love to see people dress the cables in the back of a cabnet with uniform bending radius' and with a proper service loop. I would love to see that in every data center I work in. I wouldn't have to replace cables when a channel bank gets moved. I would love to see this fiber I have to deal with be secured properly and not cinched down hard enough to break your 1st. 6th, 18th, 32nd, and 72nd strand. That would save me a lot of work. I am not going to pay to see what they say... but I bet its just a culmination of proper install techniques and "industry standard" dressings to make the tech's and the installers times easier. Remeber "if it aint pretty, it dont work", err... that's what they told me in BICC anyways.

This is a really bad idea, sort of

[DaEMoN128]

From TFA, this is a "checklist" for CIO's. Last thing I need is my PHB having a list to check off and thinking they are requirements instead of suggestions. You never give a PHB this much info. They dont know what you are doing a good part of the time anyways... and a little info is more dangerous than none.

It can be a good idea if the techs get a hold of it though and stop giving my 2 inches of slack on these fiber runs and give be a proper service loop with good cable dressing instead of the rats nests I've had to fix recently.

Re:But of course

[frovingslosh]

Not sure about you, but if I was setting up a data center, I'd fork the $250. It'd probably be your smallest expense and is bound to have some good ideas in it.

Well that sure is an interesting comment. Just how many marks do you think that you can round up who would each pay $250 a pop for a 148 page report because it "is bound to have some good ideas in it"? At that price it wouldn't take too many people to convince me to crank out 150 pages of opnion, particular when the subject is as broad as "every single aspect of a data center". Maybe even sell it at the bargan price of $229.99 and undercut this report (but be close enough in price that some might think mine and the one discussed here are the same). And it wouldn't be like I was stealing their money, would it? It's bound to have some good ideas in it.

1 shoe does not fit all

[zenst]

Being unable to lash out and indeed darned if I woudl ever pay $250 for a datacentre for dummies kidna guide given that every case has to be treated on its own merets.

I wonder if they cover aspects like power phase balancing given alot of places have 3-phase and we all know how box's move about so the aspect of auditing the balancing across the 3 phases comes in. Why well power costs given you pay for the highest usage over the 3 and then there is the UPS aspects and resiliance aspects.

Oh and DR sites, if you have enough eggs for one basket then you need backup plans.

As you who work in the feild can imagine there are sooo many aspects that need to be looked at and whilst a guidline/crib sheet is always welcome no two datacenters are the same or some manufactuirer would be making a killing selling them in a box ;0.

The smallest deatil on one site becomes the largest priority on another. Hell i've even set up a site were we had critical servers spread across the area just to give that extra resiliance incase a train derailed and came thru, albeit remote possibility but still was a potential, and all before 9/11 which in itself shows the level of which murphy's law can stoop.

So whilst this type of thing is nice I do erk when people hold it up as a definitive guide as the only people fooled by that are those that dont know and highup managers and we all know a little knowledge is a dangerous thing.

Still its nice to have stuff like this as long as they are viewed in the context of guidlines and not guides in themselves.

I mean if all doctors were to follow a needle injection guidline of insert needle in left arm 13cm down from main index finger and on the horizontal plane of the wrist joint, not everybody would get the injection into a vien as we are all not exactly the same albiet similiar and indeed some people have no index finger nor left arm.

I personaly wish they would invest more time in a cable standard were the outer colouring deformed in relation to the number of times the cable is bent, now that would in itself be a much more useful input into datacenters in that they could pre-emptivly identify cable that is going past its shelf life or been over abused/used and liable to induce errors.

April's fool?

[VincenzoRomano]

The document TIA-842 has been issued on April 1, 2005!
I'd not take in too serious consideration this doc!

Re:Link to the document

[jrockway]

Sample scenario:

Hi, your datacenter isn't TIA-compliant. We won't sell you xyz hardware. We won't sell you xyz fire-suppression system.

Hi, I notice your datacenter doesn't have a fire-suppression system. I have to close it, by law, until it's installed.

Hi, I can't install a fire-suppression system until you bring the datacenter up to TIA-standards.

Needless to say, TIA doesn't have to make their spec law for it to be able to screw your datacenter over.

Re:The three steps

[saintp]

Don't worry, it's available in PDF. I predict it's available over eDonkey or BitTorrent within the week.

Purpose of the Datacenter

[PhYrE2k2]

But when you have a formal standard, you have something to measure against. Every aspect of the data center design is not only standardized, but the how's, why's and therefore's are spelled out.

But not all datacenters are equal for a reason. I've seen maybe 20-30 datacenters in the past few years for various clients and they all have different features, different offerings, and different goals.

I'll list a few of the big differences I've seen in my experiences:

Some want to be in the downtown core, close to many businesses, but charging a premium for the space. Others claim that being in the outskirts of the city provide security in the event of any problems (mainly hyped due to 'terrorist attacks').

Some feel the need for N+2 generators, others more. Some feel that a fallback to city power if their PDUs ever fail is good, and others feel there should be a whole other protected power distribution system (at an extremely high cost for something rarely used).

Some like cooling each rack from the top, others blow it up every other isle and suck air down on the opposite. Some cool the whole room, claiming lots and lots of cooling units around the outside does the trick.

Some like the datacenter two stories underground. Others claim that they're a first target for flooding and other problems stereotypically associated with a basement. Others say that the datacenter on the 10th floor of a tower is inaccessable and subject to other security feats of the building.

Some like dedicated buildings, others like quietly slotting themselves in office towers.

A few I went to were monitored from 3000+km's away, and others had 24/7 onsite staff. Some had technical electronic keys, and others a simple mailbox key. Some had biometrics, others just a key.

One I went to even had outter walls capable of withstanding most missiles. Others had windows with only paper over them for security reasons.

Some let you roam freely by a security personelle and simply log equipment, others weigh you on the way out to make sure that you didn't take anything you didn't show up with without signing it out.

The point is each of these serves a very different purpose. If you are going to have lots of untrusted people working on equipment, it's important to make sure nobody takes anything. Each one has its advantage and disadvantage, and I don't think any one of them is 'right'- it's just trying to find a solution to problems that experience has provided.

Is there a right answer with anything? Who is to say that any answer is right or wrong? They're just different solutions to the problem. If power stays up, systems are secure, systems get cooled, and the network is available, who is to say the solution is wrong?

-M

Re:But of course

[dtfinch]

To me, that does not imply buying a cheap rack, getting a T1 and a router, and sticking 5 servers in whatever closet is available in your office space.

You just described our setup almost exactly.

The Basics....

[Duncan3]

Above sea level. Check.
Somewhere the US/Chinese government won't be monitor ing everything... still looking.