Slashdot Mirror
Raised Flooring Obsolete or Not?
mstansberry writes "In part three of a series on the price of power in the data center, experts debate the merits of raised flooring. It's been around for years, but the original raised floors weren't designed to handle the air flow people are trying to get out them today. Some say it isn't practical to expect air to make several ninety-degree turns and actually get to where it's supposed to go. Is cooling with raised floors the most efficient option?"
Where else am I going to store my beer so it can stay cold and the boss not find it?
But then where will we keep the bodies?
As long as the space under the floor has a negative or positive atmosphere I can't see how somme turns have anything to do with the air flow.
Says that raised floors may be inefficient if it gets block. Then says alternatives are expensive. Direct AC where you need it, the article says.
Why wouldn't raised floors be bad if you used them properly?
A NYC lawyer blogs. http://www.chuangblog.com/
I thought the raised flooring was just to make the people working there look taller and more impressive, kinda like how they do with pharmacists.
Another big reason for raised floors is to handle wiring. I know companies where it was installed only for this reason. Cooling wasn't even on their minds.
Proverbs 21:19
I interned at ARL inside of Aberdeen Proving Grounds this past summer and when touring the supercomputer room (more like cluster room these days), the guide said they used one of the computers in the room to simulate the airflow in that room so they could align the systems for better cooling. How geeky is that!
It's in lower power chips, more efficient PSUs, and possibly liquid cooling where the radiator is outside the building (or a heat exchanger to heat pump loop in hot climates).
To paraphrase a popular saying: "It's the COMPUTERS, stupid!"
Inefficient architectures must be discarded to make way for more modern, smaller, COOLER processors.
Let's address the real problem here -- not the SYMPTOM of hot air.
We need to address the COMPUTERS.
If you "get" pointers add me as a friend (116)!
I am waiting for the day where someone invents a computer that doesn't need to be cooled or generate excess heat.
Think about the lightbulb....A standard 60-watt incadescent bulb generate lots of heat. A better design is something like the LED bulbs that generate the same amount of lumens, with much less power, and more importantly little to no heat.
Good design can allow these devices to not generate excess heat, hence eliminating the need for the raised floor.
Just have the whole data center submerged in an inert solution like the one made by 3M (fluorinert?), and have the workers wear scuba equipment.
Most. Efficient. Cooling. Evar!
Someone needs to create an air interconnect standard that lets server room designers snap-on cold air supplies onto a standard "air-port" on the box or blade. The port standard would include several sizes to accomodate different airflow needs and distribution form large supply ports to a rack of small ports on servers. A Lego-like portfolio of snap-together port connections, tees, joints, ducts, plenums, etc. would let an IT HVAC guy quickly distribute cold air from a floor, wall or ceiling air supply to a rack of servers.
Two wrongs don't make a right, but three lefts do.
If something is airtight, putting air in one end will move air out the other end.
The problem lies with larger datacenter environments. Imagine a room the size of a football field. Along the walls are rows of air conditioners that blow cold air underneath the raised floor. Put a cabinet in the middle of the room and replace the tiles around it with perforated ones and you get a lot of cooling for that cabinet. Now start adding more rows & rows of cabinets along with perforated tiles in front of each of them. Eventually you get to a point where very little cold air makes it to those servers in the middle of the room because it's flowing up through other vents before it can get there. What's the solution? Removing servers in the middle of hotspots & adding more AC? Adding ducting under the floor to direct more air to those hotspots? Not very cheap & effective approaches...
it can turn on a dime, but also stay on that dime. poor circulation results. trumpets have nice (if tight) curves, and even building ducts can have redirects inside the otherwise rectangular ducts to minimize trapped airflow in corners. for the most part even those corners are curved to help the stream of air.
most server rooms aren't part of the duct, for example, the one here is large and rectangular, with enormous vents at either end. not very well designed.
airflow is a very complicated problem, my old employer had at least three AC engineers on full time staff to work out how to keep the tents cold ( I worked for a circus, hence the nick.) the ducting we had to do in many cases was ridiculous.
why do you think the apple engineering used to use a cray to work out the air passage through the old macs. just dropping air-conditioning into a hot room isn't going to do jack if the airflow isn't properly designed and tuned. air, like many things, doesn't like to turn 90 degrees, it needs to be steered.
-- it's ridiculous how many people misspell ridiculous... (damn, damn, damn...)
We worked very closely with Liebert ( http://www.liebert.com/ ) when we recently rennovated our data center for a major project. The traditional CRAC (Computer Room AC) units supplying air through a raised floor is no longer viable for the modern data center. CRAC units are now used as supplemental cooling, and primarily for humidity control. When you have 1024 1U, dual processor servers producing 320 kW of heat in 1000 sq ft of space, an 18 inch raised floor (with all kinds of crap under it) is not adequate to supply the volume of air needed to cool that much heat in so small a space.
We had intended to use the raised floor to supply air, but Liebert's design analysis gave us a clear indication of why that wasn't going to work. We needed to generate air velocities in excess of 35 MPH under the floor. There were hotspots in the room where negative pressure was created and the air was actually being sucked into the floor rather than being blown out from it. So, we happened to get lucky as Liebert was literally just rolling off the production line their Extreme Density cooling system. The system uses rack mounted heat exchangers (air to refrigerant), each of which can dissipate 8 - 10 kW of heat, and can be tied to a building's chilled water system, or a compressor that can be mounted outside the building.
This system is extremely efficient as it puts the cooling at the rack, where it is needed most. It's far more efficient than the floor based system, although we still use the floor units to manage the humidity levels in the room. The Liebert system has been a work horse. Our racks are producing between 8 - 9 kW under load and we consistently have temperatures between 80 - 95 F in the hot aisle, and a nice 68 - 70 F in the cold aisles. No major failures in two years (two software related things early on; one bad valve in a rack mounted unit).
But it also eliminates the joy of making fun of coworkers who gets lost in a raised floor, or closing them in when they go on a hunt for something...
(S(SKK)(SKK))(S(SKK)(SKK))
So long as you have positive air pressure under your floor, you'll get *some* effect from your perf tiles. However, as I'm sure some fluid dynamics folks will jump in with, air flow is a HARD problem. Yeah, so you're getting cold air coming up through your perfs. Well, most of them. Some of them are actually pulling air DOWN. Why?
:)
If you're bored, check out TileFlow. It's an underfloor airflow simulator. You put in your AC units, perf tiles, floor height, baffles, you name it. It will (roughly) work out how many CFM of cold air you're going to see on a given tile. It's near-realtime (takes a second to recalculate when you make changes), so you can quickly add/remove things and see the effect. I spent some time messing with this a couple of years ago, and it's very easy to set up a situation where you have areas in your underfloow with *negative* pressure.
The article basically summed it up for me:
McFarlane said raised floors should be at least 18 inches high, and preferably 24 to 30 inches, to hold the necessary cable bundles without impeding the high volumes of air flow. But he also said those levels aren't realistic for buildings that weren't designed with that extra height.
I'd go with 24 inches MINIMUM, myself. Also, proper cable placement (ie: not just willy-nilly) goes a long way towards helping airflow issues. Like they said though, you don't always have the space.
Of course, with the introduction of a blade chassis or 4, you suddenly need one HELL of a lot more AC
Endless arguments over trivial contradictions in books written by ignorant savages to explain thunder in the dark.
When you are designing for a space (such as a room) you design for the shortest amount of ductwork for the greatest amount of distribution. Look up in the ceiling of an office complex sometime and count the number of supply and return diffusers that work to keep your air in reasonable shape. All of the ducts that supply this air are smooth, straight and designed for a minimal amount of losses.
All air flow is predicated on two imporant points within a given pipe (splits and branching with in the duct work is not covered here): pressure loss within the pipe and how much power you have to move the air. The higher the pressure losses, the more power you need to move the same amount of air. Every corner, turn, rough pipe, longer pipe all contribute to the amount of power needed to push the air through at the rate you need.
Where am I going with all of this? Well under floor/raised floor systems do not have alot of space under them and it is assumed that the entire space under it is flexible and can be used (ie no impediments or blockages). Ductwork is immobile and does not appreciate being banged around. Most big servers need immense amounts of cooling. A 10"x10" duct is good for roughly 200 CFM of air. That much air is good for 2-3 people (this is rough, since I do not have my HVAC cookbook in front of me.. yes that is what it is called). Servers need large volumes of air and if that ductwork is put under the floor, pray you don't need any cables in that area of the room. Before you ask: Well why don't we just pump the air into the space under the floor and it will get there? Air is like water, it leaves through the easiest method possible. Place a glass on the table and pour water on the table and see if any of the water ends up in the glass. Good chance it ends up spread out on the floor where it was easiest to leak out. Unless air is specifically ducted to exatcly where you want it, it will go anywhere it can (always to the easiest exit).
Ductwork is a very space consuming item. Main trunks for 2 and three story buildings can be on the order of four to five feet wide and three to four feet high. A server room by itself can require the same amount of cooling as the rest of the floor it is on. (ignoring wet bulb/dry bulb issues, humidity generation and filtering, we are just talking about number of BTUs generated). A good size server room could easily require a seperate trunk line and return to prevent the spreading of heated air throughout the building (some places do actually duct the warm air into the rest of the building during the winter). Allowing this air to return into the common plenum return will place an additional load on the rest of the buildings AC system. Place the server on a seperate HVAC system to prevent overloading the rest of the building's AC system (which is designed on a per square foot basis assuming for a given number of people/computers/lights per square foot if the floor plan does not include a desk plan layout).
Architectural plans are like computer source code with a couple of differences: You only compile once.
That only works until you have a situation where you need to cut the green wire with the yellow stripe, NOT the black wire with the white stripe, in order to shut down your server before it explodes. That oxygenated fluid is pink, making colour detection damn near impossible.
Now, if you're willing to host an alien spaceship at the bottom of your datacentre, maybe they could lend a hand...
Endless arguments over trivial contradictions in books written by ignorant savages to explain thunder in the dark.
Raised flooring is useful for several reasons, moving cool air through a data center is only one of them. While requiring air to make severe turns to get out of the floor isn't optimal, most cabinets and the equipment in those cabinets is engineered with this in mind. Air is generally drawn in through the front of the cabinet and device and warm air blows out the back. Fans in the equipment pull the air in - the air doesn't have to "turn" on its own again (not that is really did in the first place). Warm air then rises after leaving the device where it is normally drawn back into the top of the AC unit.
Raised flooring also provides significant storage for those large eletrical "whips" where 30A (in most US DCs any how) circuits are terminated as well as a place to hide miles of copper and fiber cable (preferably not too close to the electrical whips). Where else would you put this stuff? With high density switches and servers, we certainly aren't seeing less cable needed in the data centers. Cabinets that used to hold five or six servers now hold 40 or more. Each of these needs power (typically redundant) and network connectivity (again, typically redundant), so we actually have more cables to hide than ever before.
Cabinets are built with raised flooring in mind. Manufactureres expect your cabling will probably feed up through the floor into the bottom of the cabinet. Sure, there is some space in the top of the cabinets, but nothing like the wide open bottom!
Anyhow, there you have the ideas of someone who is quickly becoming a dinosaur (again) in the industry.
Okay, screw this post about putting the servers in a giant tank filled with a coolant. Put the servers in a vertical wind tunnel so you can practice your sky diving while swapping a hard disk!
God is dead -- Nietzsche
Nietzsche is dead -- God
Zombie Nietzsche lives! -- Zombie Nietzsche
Another big reason for raised floors is to handle wiring.
or pluming. I'm serious. (An a bit OT)
When I was at IBM's Cottle Rd. facility, now (mostly) part of Hitachi, they had just finished rebuilding their main magnetoresitive head cleanroom (Taurus). They took the idea from the server techs, and dug out eight feet from under the existing cleanroom (without tearing down the building) and put in a false floor.
All of the chemicals were stored in tanks under the floor. Pipes ran veritcally, and most spills (unless it was something noxious) wouldn't shut down much of the line. It was a big risk but, if what I hear is correct, people still say it's the best idea they had in a while.
I'd rather have someone respond than be modded up.
There are a number of slashdot visitors that do actually care about server room issues. The fact that you don't understand the need does not negate it's importance.
Large organizations rely on server rooms for their computing environment. Having a cobbled environment where the file server is on the 3rd floor, and the application server is in the janitor's closet, etc. is a recipe for disaster. Troubleshooting connectivity issues (among others) can end up costing more than the apparent simplicity of such a design.
Understanding ways to better cool the space that our servers occupy is important. And being able to do so in a cost effective manner is also important. The organization that I work in has one in-house server room (containing 60 racks of servers), and one 'co-located' server room (containing 72 racks of servers). Heat and power are the two killers. If we experience a 50% power loss (assume that one power grid is knocked out), do we have enough power to run AND cool the server room? If not, what percentage of my gear do I need to shut down in order to prevent overheating, without impacting critical business systems (like payroll).
If we can find a cheaper / better / more cost effective method for cooling that utilizes less power, or find a way to use the cooling systems that we have in a more efficient manner, is that not worth an article on slashdot?
IMHO, This is a valid topic.
Raised floor cooling was designed back when the computer room held mainframe and telephone switch equipment with vertical boards in 5-7 foot tall cabinets. The tile was holed or removed directly under each cabinet, so cool air flowed up, past the boards and out through the top of the cabinet. It then wandered its way across the ceiling to the air conditioners' intakes and the cycle repeated.
Telecom switching equipment still uses vertically mounted boards for the most part and still expects to intake air from the bottom and exhaust it out the top. Have any AT&T/Lucent/Avaya equipment in your computer room? Go look.
Now look at your rack mount computer case. Doesn't matter which one. Does it suck air in at the bottom and exhaust it out at the top? No. No, it doesn't. Most suck air in the front and exhaust it out the back. Some suck it in one side and exhaust it out the other. The bottom is a solid slab of metal which obstructs 100% of any airflow directed at it.
Gee, how's that going to work?
Well, the answer is: with some hacks. Now the holed tiles are in front of the cabinet instead of under it. But wait, that basically defeats the purpose of using the raised floor to move air in the first place. Worse, that mild draft of cold air competes with the rampaging hot air blown out of the next row of cabinets. So, for the most part your machines get to suck someone elses hot air!
So what's the solution? A hot aisle / cold aisle approach. Duct cold air overhead to the even-numbered aisles. Have the front of the machines face that cold aisle in the cabinets to either side. Duct the hot air back from the odd-numbered aisles to the air conditioners. Doesn't matter that the hot aisles are 10-15 degrees hotter than the cold aisles because air from the hot aisles doesn't enter the machines.
Moderating "-1, Disagree" is simple censorship. Have the guts to post your opinion.
Note that I'm not calling the parent poster stoopid, but rather the design of forcing cold air through the *floor*. As the parent here notes, cold air falls. This is presumably why most home fridges have the freezer on top.
I was most surprised to read this article. I've never worked in a data center, but I have worked in semiconductor production cleanrooms, and given the photos I've seen of data centers with the grated flooring, I guess I always assumed the ventilation was handled the same way as in a cleanroom -- new air in from the ceiling, old air whisked away through the floor. (This ensures that any particles, which will naturally fall if heavier than air, will be sucked out of the room.) Note that this is obviously *not* a passive system designed to use convection, but rather an active system using lots of fans.
While a passive convection system with the cold pulled up from below is a nice theory, you can run into the same problems others have pointed out -- what if the bottom units suck in all the cold air? The top units are left too warm.
Meanwhile, if you drop cold air from above, sure, the top units might suck a lot of that in -- but any cold air that isn't sucked in will naturally continue to drop relative to warmer air, ensuring that the lower units are not cooked. If you want to be especially careful about it, you could route all the cold air outputs towards the perimeter of the room and put the uptakes in the center of the ceiling to ensure a vortical flow.
Just my ¥2.
"What in the name of Fats Waller is that?"
"A four-foot prune."
Well, you're close. You are correct that the answer lies in a "hot aisle/cold aisle" configuration. The difference is, it works better when the cold air is coming up from below the raised floor tiles.
Why? You must keep in mind, you're not trying to pump "cold" air in, you're trying to take heat out, and as Mother Nature knows, heat rises. So why not harness the natural convection of heat, allow it to flow up to the ceiling, and have some "perf" ceiling tiles and use the space over the ceiling tiles as your return plenum. Thus, you end up with a positive pressure beneath the raised floor, your heat load in the data room, and your negative pressure over the ceiling tiles leading back to your CRAC units.
I assure you it works fabulously in our 2 year old data center at a major financial company. The other advantage to raised floor is, you don't have to worry about water being overhead. No one wants a condensor water, chilled water, or glycol pipe bursting over a row of server racks. But, put your power whips in liquid-tite conduit, use cable racks for the CAT5 and there's no problem if you have a leak. The leak detection ropes pick it up and you can contain it before it becomes a problem.