Today, I was sitting at my desk busy with code for a new app when I'm handed a card. Inside are all of my co-worker's signatures, saying thanks for all the hard work that I've done over the past year.
Unfortunately, the card went on to say "Good luck in your future endeavors...."
In that particular data center, we're using Liebert System 3's with the water coils. They get their chilled water from 2 (1 being redundant) Trane chillers.
Having said that, in a previous incarnation, at a much larger datacenter, we had mostly the standard Liebert System 3's with a few Data-Aire's peppered throughout. I have to say that overall, I've been pretty happy with the Lieberts, they are generally well designed units, and they run forever.
Well, you're close. You are correct that the answer lies in a "hot aisle/cold aisle" configuration. The difference is, it works better when the cold air is coming up from below the raised floor tiles.
Why? You must keep in mind, you're not trying to pump "cold" air in, you're trying to take heat out, and as Mother Nature knows, heat rises. So why not harness the natural convection of heat, allow it to flow up to the ceiling, and have some "perf" ceiling tiles and use the space over the ceiling tiles as your return plenum. Thus, you end up with a positive pressure beneath the raised floor, your heat load in the data room, and your negative pressure over the ceiling tiles leading back to your CRAC units.
I assure you it works fabulously in our 2 year old data center at a major financial company. The other advantage to raised floor is, you don't have to worry about water being overhead. No one wants a condensor water, chilled water, or glycol pipe bursting over a row of server racks. But, put your power whips in liquid-tite conduit, use cable racks for the CAT5 and there's no problem if you have a leak. The leak detection ropes pick it up and you can contain it before it becomes a problem.
Engineers at NASA were quick to dispel rumors that there had been a communication breakdown within their organization. Reached for comment at their 2 bedroom apartment, between shifts at Wal-Mart, Air Supply, http://www.airsupply-online.com/ said they were "deeply saddened". Associates at Amazon.com stated that they "only ship what was ordered" and take no responsibility at the misunderstanding. Developing.....
UPS's do not run exclusively off of DC. You are correct that they convert AC to DC, then route it through the battery strings, then invert it back to AC current. While this does generate some heat, it is NOTHING compared to the server racks. I've worked in datacenter environments for several years now, and I can say that one of the biggest foes to efficient cooling is poor space planning.
I've never seen people so difficult to communicate with as hardware planning people. You would be amazed at how much better a computer room gets cooled when the computer equipment gets installed properly in a "hot aisle/cold aisle" configuration. Also, vendors and hardware folks don't like to have things pointed out that they're not doing, like making sure not to install a top discharge cabinet on the edge of a cold aisle right next to a front intake cabinet, or installing plenums inside the cabinets as some vendors recommend.
A combination of good space/hardware planning as well as honesty and communication in determining potential heat loads are probably the 2 biggest factors in keeping a computer space cold, IMHO. No one's being helped by just guessing as what a rack full of SunFire servers is going to put out in terms of heat, find out from the manufacturer. And don't feel that your engineering staff is trying to tell you how to do your job or piss you off by letting you know that a rack you've installed is disrupting airflow. We're all in this together, remember?
I must say, as a Critical Facilities Engineer, I feel that most of the posters thus far are drastically downplaying (intentionally or not) the complexity of modern facility management as well as BAS systems.
I work for a large commercial real estate firm at a campus for a very large financial institution. Our facility is just over 1,000,000 sq feet and is comprised of 6 buildings including a data center. It is my opinion that the people that "take over" management and implementation of BAS's as they move forward can only be an as yet unkown hybrid of Facility Engineers and IT savvy people.
The reality is, there are many more things controlled and monitored by a BAS than just lights or a few VAV's to cool an office. Modern HVAC systems are quite complex and need to take into account hundreds of factors such as outside air temperature, drybulb and wetbulb temperature, relative humidity, static duct pressure, variable frequency drives on condenser water pumps, etc. (basically, it's much more complicated than "turning up the heat").
Additionally, the task of making staunchly built, proprietary communication protocols (i.e. Wonderware, Liebert, PLC's) talk to one another in a language/at a speed that each can understand is assuredly going to be an uphill battle at best.
I would argue that neither the current Engineering industry nor the current IT industry is fully capable of handling the task of taking this part of Building Automation where it needs to go. I think it's going to take people putting their egos in check, and perhaps risking the self-preserving, "essential cog" reputation earned by keeping information guarded if this is going to come to fruition.
Already we're able to do amazing things with our BAS (Insight by Siemens in case anyone's wondering) and I, as a confessed geek, am excited to see what more we'll be able to do. I think it's great that I can roll my chiller banks from the lead to the lag from a laptop on my kitchen table without the client being able to notice anything at all. I also think it's pretty amazing that if a static switch sees an irragularity on a wave form coming off of a UPS system's battery string, that I can be alerted of it, as well as have a record of it for trending/troubleshooting purposes. Say what you will, but I definitely think this is going to be an area to watch, and I dare say, probably one that will start in the commercial sector and quickly move into the consumer group. Think how amazing it will be to be able to turn the lights on at your house from your PC at work before you leave to drive home or to have your home HVAC equipment adapt to outside conditions or upcoming weather reports available online. I know there are some "home automation" products available now, but aside from webcam monitoring and a few light controlling relays, there really isn't anything exciting yet. I think it's going to get really interesting, and I think BAS systems are going to drive it.
I was also wondering where RPG was. Obtuse a language as it was/is, I was interested to see where it's development began. I had the displeasure of using this language in school and truly grew to hate it and all it's evil "indicators".
I feel sorry for anyone who had to use this language for any length of time particularly in a work environment. I don't think any programmer would miss having to manually set various indicators (like LR) to "on" just to properly terminate a program, and all the other quirks of RPG.
I'm still not sure I get it. Specifically, I guess I don't get "guitar-bot". As a reasonably accompomplished guitar player myself, I was pretty interested in hearing what the guitar robot could do. Instead of being the fairly organic yet dissonant sounds as the piano and pipe organ pieces were, it sounded like one of those old film strip soundtracks that they used to show in science class or driver's education classes (you remember, think soundtrack to "Blood on the Asphalt"). Then again, the version of "Yesterday" sounds like you're listening to the Beatles record after eating mushrooms but that's just me.
I should begin by saying that I am not a sys admin. I am, however, a critical facilities engineer who until very recently worked at a Tier I data center for a VERY large Texas based technology company (think Ross Perot).
I am in agreement with the assertion that LJ is responsible for their outage by failing to either inspect what their Facilities people are assuring them of, or failing to do it themselves if they're operating their own facility.
The data center I worked at also had 2 redundant power feeds from different grids, so if 1 went down, we had another. Each of these fed 2 seperate switchgears at either "end" and were seperated by a Tie-Breaker, so the gear could be fed by either source if the other were to fail.
We had 2 redundant UPS systems, each comprised of 4 modules, 1 of which was also redundant. Each module had its own dedicated string of wet cell batteries. This meant that if 1 entire UPS system went down, we could stand on the other one. It also meant that we could migrate the building load to one UPS so we could PM the other one. The redundant module was also significant. If in the unlikely event that we were to lose 1 entire UPS, we could still shift the load around on the remaining UPS modules (while online) in the event of a component failure on one of the modules.
The critical load (the servers, tape silos, mainframes, etc) were all fed from PDU's, which were each fed from Static Transfer Switches. These Static switches were "make before break" using SCR's, so in the event of a loss of one source, you could roll to the other source transparenlty with regard to the computers. Also, a lot of the newer server racks are made to be "dual fed". That means, even if the entire system works properly, but the PDU (power distribution unit) from which the server gets its power were to fail, the server is still fed from another PDU (which in turn is fed from the aforementioned redundancies) so failure is EXTREMELY unlikely.
And yes, we had 6 diesel generators. 3 Caterpillar (much more reliable) and 3 Cummings. The building could run off of 3 (2 in a real crisis with some load shedding), and each generator had its own 6,000 gallon underground fuel tank.
We tested each of these components regularly and meticulously. Once a month, we rolled the entire building load to the generators for 2 hours. We did breaker testing of ALL critical breakers annually. We did Infrafred Scanning of all breakers, swtichgear, tranformers, transfer swtiches, etc annually looking for sources of heat, indicating potential problems. We load banked our generators annually, running them at 100% for 4 hours each. We had regularly scheduled PMs on EVERY piece of critcal gear, and guaranteed 100% uptime.
My point is, it is MUCH more complex than a UPS just "blowing up". The many layers of redundancy should be more than adequate to handle a catastrophic event. If any of the components were to blow up, (i.e. Static Switch, UPS module, battery string, Main Breaker) there should be a redundant one in place to assume duty. Additionally, I have to wonder why the generators didn't fire up (I'm assuming they didn't if the site was down for so long). Anyone who would invest the time and money to put in this kind of infrastructure and not do regular testing of each component is an idiot, plain and simple. Sure, there could have been extenuating circumstances, but when you're in Critical Facilities Management, your job is to think of and plan for these things before they happen. No, you can't prevent shit from happening, but you can make it very unlikey that shit will happen to all of your equipment at the same time.
Just my 2 cents, thought it might be interesting.
Slashdot Mirror
Hmmm, GNU=Gnu is Not Unix......now what people have the nerve to use recursive acronyms? "Oh hello, pot, my name's kettle, and you're black".
How can your web page miss with this winner?
Since when do these guys know anything about astrophysics?
Today, I was sitting at my desk busy with code for a new app when I'm handed a card. Inside are all of my co-worker's signatures, saying thanks for all the hard work that I've done over the past year.
Unfortunately, the card went on to say "Good luck in your future endeavors...."
(Pushing Tin Quote)
"There is a finite amount of manpower available to do this work"
"Uh oh, looks like you're going down the tubes."
"What? I'm not going down the tubes!"
"Yeah, it happens every time you use the word FINITE."
In that particular data center, we're using Liebert System 3's with the water coils. They get their chilled water from 2 (1 being redundant) Trane chillers.
Having said that, in a previous incarnation, at a much larger datacenter, we had mostly the standard Liebert System 3's with a few Data-Aire's peppered throughout. I have to say that overall, I've been pretty happy with the Lieberts, they are generally well designed units, and they run forever.
Well, you're close. You are correct that the answer lies in a "hot aisle/cold aisle" configuration. The difference is, it works better when the cold air is coming up from below the raised floor tiles.
Why? You must keep in mind, you're not trying to pump "cold" air in, you're trying to take heat out, and as Mother Nature knows, heat rises. So why not harness the natural convection of heat, allow it to flow up to the ceiling, and have some "perf" ceiling tiles and use the space over the ceiling tiles as your return plenum. Thus, you end up with a positive pressure beneath the raised floor, your heat load in the data room, and your negative pressure over the ceiling tiles leading back to your CRAC units.
I assure you it works fabulously in our 2 year old data center at a major financial company. The other advantage to raised floor is, you don't have to worry about water being overhead. No one wants a condensor water, chilled water, or glycol pipe bursting over a row of server racks. But, put your power whips in liquid-tite conduit, use cable racks for the CAT5 and there's no problem if you have a leak. The leak detection ropes pick it up and you can contain it before it becomes a problem.
Curse you, now I have to go around singing Mr. Mister all day. I am so tortured.
Engineers at NASA were quick to dispel rumors that there had been a communication breakdown within their organization. Reached for comment at their 2 bedroom apartment, between shifts at Wal-Mart, Air Supply, http://www.airsupply-online.com/ said they were "deeply saddened". Associates at Amazon.com stated that they "only ship what was ordered" and take no responsibility at the misunderstanding. Developing.....
No, Competence does not mean competition. Please disregard.
UPS's do not run exclusively off of DC. You are correct that they convert AC to DC, then route it through the battery strings, then invert it back to AC current. While this does generate some heat, it is NOTHING compared to the server racks. I've worked in datacenter environments for several years now, and I can say that one of the biggest foes to efficient cooling is poor space planning.
I've never seen people so difficult to communicate with as hardware planning people. You would be amazed at how much better a computer room gets cooled when the computer equipment gets installed properly in a "hot aisle/cold aisle" configuration. Also, vendors and hardware folks don't like to have things pointed out that they're not doing, like making sure not to install a top discharge cabinet on the edge of a cold aisle right next to a front intake cabinet, or installing plenums inside the cabinets as some vendors recommend.
A combination of good space/hardware planning as well as honesty and communication in determining potential heat loads are probably the 2 biggest factors in keeping a computer space cold, IMHO. No one's being helped by just guessing as what a rack full of SunFire servers is going to put out in terms of heat, find out from the manufacturer. And don't feel that your engineering staff is trying to tell you how to do your job or piss you off by letting you know that a rack you've installed is disrupting airflow. We're all in this together, remember?
I must say, as a Critical Facilities Engineer, I feel that most of the posters thus far are drastically downplaying (intentionally or not) the complexity of modern facility management as well as BAS systems.
I work for a large commercial real estate firm at a campus for a very large financial institution. Our facility is just over 1,000,000 sq feet and is comprised of 6 buildings including a data center. It is my opinion that the people that "take over" management and implementation of BAS's as they move forward can only be an as yet unkown hybrid of Facility Engineers and IT savvy people.
The reality is, there are many more things controlled and monitored by a BAS than just lights or a few VAV's to cool an office. Modern HVAC systems are quite complex and need to take into account hundreds of factors such as outside air temperature, drybulb and wetbulb temperature, relative humidity, static duct pressure, variable frequency drives on condenser water pumps, etc. (basically, it's much more complicated than "turning up the heat").
Additionally, the task of making staunchly built, proprietary communication protocols (i.e. Wonderware, Liebert, PLC's) talk to one another in a language/at a speed that each can understand is assuredly going to be an uphill battle at best.
I would argue that neither the current Engineering industry nor the current IT industry is fully capable of handling the task of taking this part of Building Automation where it needs to go. I think it's going to take people putting their egos in check, and perhaps risking the self-preserving, "essential cog" reputation earned by keeping information guarded if this is going to come to fruition.
Already we're able to do amazing things with our BAS (Insight by Siemens in case anyone's wondering) and I, as a confessed geek, am excited to see what more we'll be able to do. I think it's great that I can roll my chiller banks from the lead to the lag from a laptop on my kitchen table without the client being able to notice anything at all. I also think it's pretty amazing that if a static switch sees an irragularity on a wave form coming off of a UPS system's battery string, that I can be alerted of it, as well as have a record of it for trending/troubleshooting purposes. Say what you will, but I definitely think this is going to be an area to watch, and I dare say, probably one that will start in the commercial sector and quickly move into the consumer group. Think how amazing it will be to be able to turn the lights on at your house from your PC at work before you leave to drive home or to have your home HVAC equipment adapt to outside conditions or upcoming weather reports available online. I know there are some "home automation" products available now, but aside from webcam monitoring and a few light controlling relays, there really isn't anything exciting yet. I think it's going to get really interesting, and I think BAS systems are going to drive it.
Just my 2 cents.
I was also wondering where RPG was. Obtuse a language as it was/is, I was interested to see where it's development began. I had the displeasure of using this language in school and truly grew to hate it and all it's evil "indicators".
I feel sorry for anyone who had to use this language for any length of time particularly in a work environment. I don't think any programmer would miss having to manually set various indicators (like LR) to "on" just to properly terminate a program, and all the other quirks of RPG.
I'm still not sure I get it. Specifically, I guess I don't get "guitar-bot". As a reasonably accompomplished guitar player myself, I was pretty interested in hearing what the guitar robot could do. Instead of being the fairly organic yet dissonant sounds as the piano and pipe organ pieces were, it sounded like one of those old film strip soundtracks that they used to show in science class or driver's education classes (you remember, think soundtrack to "Blood on the Asphalt"). Then again, the version of "Yesterday" sounds like you're listening to the Beatles record after eating mushrooms but that's just me.
I should begin by saying that I am not a sys admin. I am, however, a critical facilities engineer who until very recently worked at a Tier I data center for a VERY large Texas based technology company (think Ross Perot). I am in agreement with the assertion that LJ is responsible for their outage by failing to either inspect what their Facilities people are assuring them of, or failing to do it themselves if they're operating their own facility. The data center I worked at also had 2 redundant power feeds from different grids, so if 1 went down, we had another. Each of these fed 2 seperate switchgears at either "end" and were seperated by a Tie-Breaker, so the gear could be fed by either source if the other were to fail. We had 2 redundant UPS systems, each comprised of 4 modules, 1 of which was also redundant. Each module had its own dedicated string of wet cell batteries. This meant that if 1 entire UPS system went down, we could stand on the other one. It also meant that we could migrate the building load to one UPS so we could PM the other one. The redundant module was also significant. If in the unlikely event that we were to lose 1 entire UPS, we could still shift the load around on the remaining UPS modules (while online) in the event of a component failure on one of the modules. The critical load (the servers, tape silos, mainframes, etc) were all fed from PDU's, which were each fed from Static Transfer Switches. These Static switches were "make before break" using SCR's, so in the event of a loss of one source, you could roll to the other source transparenlty with regard to the computers. Also, a lot of the newer server racks are made to be "dual fed". That means, even if the entire system works properly, but the PDU (power distribution unit) from which the server gets its power were to fail, the server is still fed from another PDU (which in turn is fed from the aforementioned redundancies) so failure is EXTREMELY unlikely. And yes, we had 6 diesel generators. 3 Caterpillar (much more reliable) and 3 Cummings. The building could run off of 3 (2 in a real crisis with some load shedding), and each generator had its own 6,000 gallon underground fuel tank. We tested each of these components regularly and meticulously. Once a month, we rolled the entire building load to the generators for 2 hours. We did breaker testing of ALL critical breakers annually. We did Infrafred Scanning of all breakers, swtichgear, tranformers, transfer swtiches, etc annually looking for sources of heat, indicating potential problems. We load banked our generators annually, running them at 100% for 4 hours each. We had regularly scheduled PMs on EVERY piece of critcal gear, and guaranteed 100% uptime. My point is, it is MUCH more complex than a UPS just "blowing up". The many layers of redundancy should be more than adequate to handle a catastrophic event. If any of the components were to blow up, (i.e. Static Switch, UPS module, battery string, Main Breaker) there should be a redundant one in place to assume duty. Additionally, I have to wonder why the generators didn't fire up (I'm assuming they didn't if the site was down for so long). Anyone who would invest the time and money to put in this kind of infrastructure and not do regular testing of each component is an idiot, plain and simple. Sure, there could have been extenuating circumstances, but when you're in Critical Facilities Management, your job is to think of and plan for these things before they happen. No, you can't prevent shit from happening, but you can make it very unlikey that shit will happen to all of your equipment at the same time. Just my 2 cents, thought it might be interesting.