Slashdot Mirror


How Do You Evaluate a Data Center?

mpapet writes to ask about the ins and outs of datacenter evaluation. Beyond the simpler questions of physical access control, connectivity, and power redundancy/capacity and SLA review, what other questions are important to ask when evaluating a data center? What data centers have people been happy with? What horror stories have people lived through with those that didn't make the cut?

211 comments

  1. Just off the top of my head by Critical+Facilities · · Score: 4, Insightful

    Beyond the simpler questions of physical access control, connectivity, and power redundancy/capacity and SLA review

    Well first of all, I don't know that I'd write any of those things off as "simple". But some other points worth looking into would be:

    1. Raised Floor Height
      Cable Management (over or under floor)
      Cooling Capacity and Redundancy
      Power Quality (not just redundancy)
      Age and Condition of Electrical Hardware (ATSs, STSs, UPSs, Generators)
      Outage/Uptime History
      Fire Suppression System and Smoke Detection System
      Maintenance records
      Maintenance records
      Maintenance records
    1. Re:Just off the top of my head by jeffmeden · · Score: 3, Insightful

      Add to that:

      -KW deliverable to each rack

      -Ambient temperature in the cold aisle and how closely it's held (and possibly make it part of SLA)

      -On site technicians (and/or security) and their hours

      -Customer access policy and applicable hours (are you going to be happy, AND are threats going to be kept out?)

    2. Re:Just off the top of my head by whoever57 · · Score: 3, Interesting

      That's interesting, but the OP really needs to know what is good or not. For example, you state "Raised Floor Height". What is good? Newer datacenters don't have raised floors because it is more energy efficient to have concrete floors. "Cooling Capacity" -- what's good and what is bad? How is this measured? Some datacenters may talk aobut how cool they keep the ambient air, but there isn't much evidence that this actually provides a noticable difference to the lifetime or any other factor related to the equipment.

      --
      The real "Libtards" are the Libertarians!
    3. Re:Just off the top of my head by Fulcrum+of+Evil · · Score: 1

      Good question - I would expect it's mostly relevant for those DCs that do underfloor cabling. Cooling capacity is measaured in kW, I believe - the ability to remove heat. since air has a low specific heat, it shouldn't matter much, so long as the temp is stable and it isn't very humid.

      --
      "We returned the General to El Salvador, or maybe Guatemala, it's difficult to tell from 10,000 feet"
    4. Re:Just off the top of my head by Triela · · Score: 1, Insightful

      Once you have assessed these technical points to your satisfaction, I think customer support's ability to communicate issues to you as they arise is the final bridge. Every datacenter will at the very least experience minor problems from time to time, and if you're not able to speak directly with the techs working the problems or if first-line customer support does not have ready access to the details of the resolution process, it sure is frustrating to be left in the dark in the meantime.

    5. Re:Just off the top of my head by JWSmythe · · Score: 5, Informative

          I noticed something when touring one datacenter. They had a neat conference room that overlooked the whole datacenter. You could see the heat rising off of one area (Google's room). They went on and on about the wonders of their cooling, and how they had so much capacity.

          We later took the guided tour. The person I was with was talking to our guide, and I was paying careful attention to our environment. There were tremendous hotspots on the floor. We're not talking about 78 degrees. It was closer to the 90's. Other spots were downright cold. Why? Because they had all this capacity, and no real planning. The circulation was insufficient, even though the capacity was available. A well populated rack will always be hot at the back, but it's expected that they will draw the air off of that area rather quickly. I've even seen datacenters that enforce their hot/cold aisles, but then there isn't much of a reason for it. There is no air return on the hot side, and it's just blowing at another aisle's cold side.

          Sometimes it's good to just walk the floor with a tech (not a salesman), and ask questions about the operation. What kind of fiber do you have coming in? How many providers? How good are your generators really? Do you test them on a regular basis? I've found a sales minion will say there are a dozen providers coming in, but it will turn out that only one has substantial fiber, and the others are sharing that. {sigh} Sometimes they will have generators, but they've never test fired them. Sometimes the tech is just frustrated at the nonsense at that datacenter, and that's indicative of how it's going to be to work with them.

             

      --
      Serious? Seriousness is well above my pay grade.
    6. Re:Just off the top of my head by Critical+Facilities · · Score: 2, Informative

      you state "Raised Floor Height". What is good?

      24" is good, 36" is better. I once had a place with 8'0".

      Newer datacenters don't have raised floors because it is more energy efficient to have concrete floors.

      Hogwash.

      Cooling Capacity" -- what's good and what is bad? How is this measured?

      Capacity is measured in BTUs, or specifically tons (12,000 BTUs to a ton). What's most important is the relationship between BTUs and KW consumptions. In a nutshell, how much heat can you remove from the building vs how much are you putting in?

    7. Re:Just off the top of my head by Sandbags · · Score: 4, Interesting

      - Raised floor is certainly important, and a given. Check
      - Cable management above AND below the floor. This is not an either-or... Check
      - Cooling capacity is hard to judge, should be scalable. Redundancy is often overlooked but is often even more important that capacity... Check
      - Power quality: never seen a big datacenter without a Liebert, or at least UPS in every rack. Power does not have the be contitioned except between the UPS and the machines/devices. A whole data center power conditioner is often more efficient, but unnecessary for the little guys. either way - check.
      - Age is irrelevent as long as it's under support. If it's not, replace it. Generators need to be run several times a year to validate their condition, and also to grease the innards... See too many good generators get kicked on and fail an hour later because the oil hand't been changed in 3 years....
      - Outages should be tracked, by system, rack row, and power distro. When system seem to be going down more frequently in one area, there's usually an underlying reason... As Google recently proved as well for us all, do not ASSUME all is well, routine disgnostics including memory scans should be performed on ALL hardware. Even ECC RAM deteriorates with age (rapidly) and needs to be part of a maintenance testing and replacement policy - Check.
      - Fire suppression is usually part of your building codes, and a given, as is the routine checks (at least anually) by law.

      In addition, we deploy:
      - Man traps on all enterences to data centers. You go in one door, it closes, then you authenticate to a second door. A pressure plate ensures only one person goes in/out at a time (and it it's tripped, a scurity guy looking at a screen has to override).
      - Full 24x7 video surveilance of the data centers.
      - in/out logs for all equipment. To take a device in/out of a datacenter requires it being logged in a book (by a designated person). This is for anything the size of a disk/tape and larger. All drive bays are audited nightly by security and if drives go missing, security reviews the access logs and server room security footage to see who might have taken them.
      - clear and consistent labeling systems for rack, shelves, cables and systems.
      - pre-cable just about everything to row level redundant switches, and have no cabling from server to other servers not passed through a rack/row switch first. Row switches connect to distro switches. This ensures cabling is simple, and predictable.
      - Colorcoded cabling: we use 1 color for redundant cabling (indicating their should be 2 of these connected to the server at all times, and to seperate cards in the backplane and seperate switches to boot), a seperate color for generic gigabit connections, another color for DS View, another color the out management network(s), another color for heartbeat cables, and yet another for non-ethernet (T1/PRI/etc). Other colors are used in some areas to designate 100m connections, special connectivity, or security enclave barriers, and non-fiber switch-to-switch connections. Every cable is labled at both ends and every 6-8 feet inbetween.
      - FULLY REDUNDANT POWER. It's not enough to have clean poewr, and good UPS and a generator. In a large datacenter (more than a few rows, or anything truly mission critical), you should have 2 seperate power companies, 2 seperate generators, and 2 fully segregated power systems at the datcenter, room, row, and rack levels. in each datacenter we use 2 Liebert mains, each row has a seperate distribution unit connected to a differnt main, and each rack has 4 PDUs (2 to each distro). Every server is connected to 2 seperat PDUs, run all the way back to 2 completely independent power grids. For a deployment of 50 servers or so this is big time overkill. We have over 3500 servers, we need this... We can not rely on a PSU failure taking out racks at a time which may server dozens of other systems each.

      --
      There is no contest in life for which the unprepared have the advantage.
    8. Re:Just off the top of my head by icebike · · Score: 3, Interesting

      Presumably the OP is looking for a hosting site, or processing center, rather than looking at purchasing the facility.

      If so very few of the items mentioned in the parent post are germane, other than Outage/Uptime History. What is under the floor is not your problem in hosting arrangement.

      You might be interested in location (flood plain, quake zone) and, but if the place has been in business for more than 10 years it all boils down to Outage/Uptime History.

      The cost, and ease of migration should the relationship sour and the names of the last big customers to exit the facility would be nice to know.

      --
      Sig Battery depleted. Reverting to safe mode.
    9. Re:Just off the top of my head by NervousWreck · · Score: 2, Interesting

      Maintenance records Maintenance records Maintenance records are Moses and all the prophets

      --
      I do not have a sig. You are hallucinating.
    10. Re:Just off the top of my head by mcrbids · · Score: 2, Interesting

      As you indicate, these are hardly simple questions!

      While I would not endorse them today, for years I hosted at GNI, part of 365 Main. Things generally worked well, even if their staff were terse and often unfriendly, so I had no particular complaints until they had a power prolem that cost us about 2000 in direct cost and about two business days to finally, fully resolve. The amount of terse double-speak that came out of them left a very bad taste in my mouth and I've left as soon as I could. Stay clear of 365 Main!

      Our new colo is Herakles Data in Sacramento. There, too, things have pretty much 'just worked', but they so much nicer to deal with! And when the inevitable downtime did happen (a 'brownout' on the part of one of their redundant Cisco routers) they were quick to explain exactly what happened and even sent us forms in case we wanted to make a claim against our SLA! (I didn't bother just because I appreciated the respect they afforded me)

      And it goes further - when I asked their sales guy about the best way to get a server ack for the development, they GAVE me one that they had replaced because of size limits for FREE! On paper, both colos are similar, with full redundant everything, plenty of certification and nice, glossy promo materials.

      In practice, they are like night and day.

      --
      I have no problem with your religion until you decide it's reason to deprive others of the truth.
    11. Re:Just off the top of my head by tempest69 · · Score: 1

      IMHO.. Check it for the standard disaster scenarios..
      Flood... is this place in new orleans, below sea level, or in Cheyenne, a mile up with 12 inches of moisture a year.
      Fire, Looting, Snow (can collapse roofs if not shoveled, and leave power outages for days). tornadoes, hurricanes, earthquakes, landslides, sinkholes, train derailment, proximity to a place someone would like to destroy (WTC, Fedral buildings,the killdozer targets). Buildings that can be affected by highway closures. Close proximity to airport increases chances of plane collision.
      See if you can get the full insurance adjustment estimates on the property. I've heard Detroit is considered the safest area for natural disasters (needs citation).
      ask rough questions of the managment. "if cogent gets in another spat with level3, will we lose any connectivity?", "can a backhoe take out out service?", "if our rival comes into the data center to physically access their boxes, how will our boxes be secure?","how much wood could a wood chuck chuck..."

    12. Re:Just off the top of my head by Anonymous Coward · · Score: 1, Interesting

      Mean height above local water table / flood plain. Inches per year of rain in the locale. Number of farmers with backhoes in the area (ever had a cable cut????).

    13. Re:Just off the top of my head by Mr.+Freeman · · Score: 1

      "Newer datacenters don't have raised floors because it is more energy efficient to have concrete floors.

      Hogwash."

      Oh, well, thank goodness we got that out of the way.

      In all seriousness though, do you have any verifiable information about this? I was under the assumption that every datacenter had a raised floor but I really don't have any idea where this idea came from.

      --
      -1 disagree is not a modifier for a reason. -1 troll, flaimbait, redundant, overrated are NOT acceptable substitutes.
    14. Re:Just off the top of my head by EdIII · · Score: 1

      A pressure plate ensures only one person goes in/out at a time (and it it's tripped, a scurity guy looking at a screen has to override).

      Uh huh. A big F&U to the guys who came up with that one. Unfortunately I always show up as "more than one person". I suspect that more than a couple people on Slashdot are similarly challenged.

      Oh yeah... the man traps. If there are going to be situations in which somebody might be trapped in there for an extended period of time have an emergency box that can be broken into with a twinkie or hot pocket or something.

      Big guys need some consideration too....

    15. Re:Just off the top of my head by Fulcrum+of+Evil · · Score: 2, Funny

      what, it's not like you're going to starve...

      --
      "We returned the General to El Salvador, or maybe Guatemala, it's difficult to tell from 10,000 feet"
    16. Re:Just off the top of my head by Anonymous Coward · · Score: 0

      And the poster said weight detection how?

      A pressure sensor doesn't have and usually doesn't include weight. It can be as simple as a grid of tiles where each tile has an on/off switch that's on whenever anyone is standing on it. So you stand on the designated pressure mat and swipe your card and enter your pin. If any of the other mats indicate something heavy is on them, the system indicates multiple people present.

      Yes, such a system can be easily fooled. A couple of *really friendly* people can possibily stand on the same mat, but it's unlikely. And no matter how overweight you may be. I seriously doubt that you "footprint" is significately larger than other people.

    17. Re:Just off the top of my head by speculatrix · · Score: 1

      * N+N redundancy in main supply *and* UPS *and* generators *and* cooling * security - access to the site, protection of the installed equipment, storing equipment being delivered * truly diverse fibre as well as sufficient bandwidth with low/no contention * good peering policies to avoid poor latency to your customers' providers * good value remote hands - some providers are very rigid about charging for every second you ask them for support, most are helpful with free basic cover * hidden charges for telephone line installations and cable management * extra charges for dual WAN uplink * cost of committed AND "overage" charges * protection from DoS attacks * rack depth - will your servers fit?! * patching infrastructure available for your own use to cable between racks if you expand, what cost? not all of the following are yes/no decisions BUT should be considered because they can make a big impact to the time taken to get your service up and running * power arrangements - who provides the PDUs - if buying yourself, will they fit, will you have the right brackets? * parking - this is not trivial in london and can be a major bug-bear if you're delivering equipment * traffic issues - if you've an emergency and you or the Dell or HP engineer can't get there to fix things without inordinate delays, you might have a growing headache * onsite facilities when working - drinks and food prep areas, bathrooms, showers even (not needed these but some sites do, if you're doing a major roll-out they could be significant) * rubbish disposal * local facilities - eateries, hotels or B&Bs * level floors - no steps - from loading bay to server floor, lifts etc. don't want to be lugging servers up/down stairs! I visited six significant colo facilities before choosing the one I picked, most had merits, some had minor (to them) problems which would have been a minor but consistent headache to me. We were recently forced to pull our entire server farm out of an inadequate colo, which cost us big time as we had to duplicate significant amount of equipment in order to maintain service instead of shutting down and relocating, as well as having to pay two hosting bills in parallel for a while. This came about because we originally chose the cheapest most convenient provider, which would have been ok for a while, but stayed with them far too long - moral, well before renewing or expanding, review whether your colo service is up to scratch.

    18. Re:Just off the top of my head by Gigabit+Switchman · · Score: 1

      This is clearly bad software design. The access card identifies you, and likely knows your weight within 10%.

      Field Guide to Datacenter Temperature Monitoring:
      http://www.sensatronics.com/index.php/support/library/265.html

    19. Re:Just off the top of my head by Cramer · · Score: 1

      Unless you're running water pipes through the floor, a few tones of concrete isn't going to do jack for data center. The reason for concrete floors is entirely engineering... cheap, strong, durable, and very easily built. It makes next to no dent in heat load. (what's the specific heat and thermal conduction rate for concrete?)

    20. Re:Just off the top of my head by speculatrix · · Score: 1

      damn, html formatting FAIL!!!

    21. Re:Just off the top of my head by Gigabit+Switchman · · Score: 1

      Do they pay attention to temperature with any granularity? "Cold aisle temperature" is not a single number; it varies. If it *doesn't* vary, they're either REALLY good (unlikely) or spending too much on cooling and charging you too much. Field Guide to Datacenter Temperature Monitoring: http://www.sensatronics.com/index.php/support/library/265.html 5% off from the Engineering department, use this code: ENG09 (expires Jan 1 2010)

    22. Re:Just off the top of my head by Trogre · · Score: 1

      You forgot the pony.

      --
      "Nine times out of ten, starting a fire is not the best way to solve the problem." - my wife
    23. Re:Just off the top of my head by Anonymous Coward · · Score: 0

      As far as the relationship between BTU's and KW, what you're really looking for is power density. Any higher tier data center is going to already have all of this calculated and should be able to dictate a power density you are required to adhere to. Density is typically expressed in Watts per Sq Foot. When touring a modern colo facility, don't be surprised to see a customer with 4 cabinets packed to the gills with high density blades in an 400-500 sqft cage. The empty space is there to stay within density limits. Ie, you may be limited to 150w/sqft. I would be hard pressed to colo my companies' gear in any datacenter/colo that couldn't articulate the density they can support. Also, for what it's worth, there was a comment a ways back about cable management both above cabinets and below the floor as being a requirement - if the colo allows cabling under-floor, make sure to check it and make sure it's clean. Even better, there shouldn't be *any* cabling underfloor - the entire purpose of a raised floor is to create a pressurized space of cooled air which is then (ideally) strategically fed to the equipment. Under-floor cabling impedes airflow, regardless of how clean it is. Raised floor is *not* intended to hide cabling! On that same page, ensure that there are tiers of above-cabinet cable management - power and data/network should be cleanly segregated.

    24. Re:Just off the top of my head by outlander · · Score: 2, Interesting

      One thing not mentioned: a rigorous procedure for handling of decommissioned equipment. Failure to have proper audit mechanisms in place for hw removal is asking for data theft.

      --
      "Truth is what works" -- William James "It works!!" -- o-dark-AM comment
    25. Re:Just off the top of my head by whoever57 · · Score: 2, Informative

      Newer datacenters don't have raised floors because it is more energy efficient to have concrete floors.

      Hogwash.

      Yeah, what do I know about the subject? I'm just quoting from a recent talk given by Subodh Bapat, Vice President, Energy Efficiency and Distinguished Engineer, Sun Microsystems.

      Oh, and there are some articles about this

      But please, continue to refute my statement with clear, unsupported, single-word denials. They carry so much weight in an argument.

      --
      The real "Libtards" are the Libertarians!
    26. Re:Just off the top of my head by orlanz · · Score: 1

      Ask for a SAS70, it will answer many questions, and at a minimum give you a very good starting point. After reading that, you might want to fill in the gaps where your own risk conclusions differ from the auditors via a visit. Additionally, I would look at the backup solutions they have, and if they have an offsite solution, then do a quick visit over there too. Also, find out how long it would take to come back online from offsite backups if Murphy strikes. I think something that a lot of people don't consider is expansion. See how much of their capacity they are already using up, how fast they can grow with your demands, and how easy/hard it would be to move out. -- Ex-Auditor.

    27. Re:Just off the top of my head by bastion_xx · · Score: 1

      I liked the Herakles center in Sacramento too. We also normally need SAS 70 Type II facilities. Those address mostly audit and security concerns, but a data center must be in good operational shape to stay current with a type II. the ones I normally run away from are central offices turned colocation. "Yeah, we are just finishing up our audit." Uh huh.

    28. Re:Just off the top of my head by Ash+Vince · · Score: 1

      On a complete tangent i have actually been trapped in a datacenter. We did not expect a repair to take as long as it did and our access expired at midnight. We tried to take a break and discover we were locked in until someone could come and open the door to renew our access cards. Only took half an hour for an engineer to attend (called out at about 1am) but was quite amusing when we joked about it after the system was up and running again.

      Certainly not a night I want to repeat in a hurry, but I had only worked for my employers for a few months so was a good way to show willing by letting my boss go home and get an early night while I supervised some engineers from a partner company who we rented space in our cabinet too.

      --
      I dont read /. to RTFA, I read /. to offend people in ignorance.
    29. Re:Just off the top of my head by Anonymous Coward · · Score: 0

      Here's a decent guide to raised floors from APC. The conclusion is basically that they aren't strictly necessary or efficient any more, but that there are still lots of old dogs around who know what do do with a raised floor.

    30. Re:Just off the top of my head by linuxwrangler · · Score: 1

      - Raised floor is certainly important, and a given. Check

      Not so fast. I've been very happy at the Switch SuperNAP which is on concrete with all cabling run overhead. And for very good reason. The typical (though changing) datacenter has mixed hot and cold air - typically cold air pumped up from the bottom (?!? kind of fighting nature, there) then allowed to rise into the ceiling. The alternative at Switch is strict hot/cold-isle isolation. Cold air drops down as per nature on the cold (intake) side and is contained on the hot-side where it rises and is pulled from the building.

      Additionally, they can handle extremely heavy equipment loads that could be difficult on raised floors.

      - Cooling capacity is hard to judge, should be scalable. Redundancy is often overlooked but is often even more important that capacity... Check

      Indeed. Many centers talk about redundant power but that is useless if the cooling goes out. High-density centers can go well past 100F in just a few minutes without cooling but I've seen SLAs that allow a several-hour cooling outage. Not to be a Switch shill (though we are happy), they have developed a setup that allows them to run indefinitely, albeit less efficiently, using air-exchange only if their cooling water supply is cut off.

      - Power quality: never seen a big datacenter without a Liebert, or at least UPS in every rack.

      I'm happy with the arrangement at Switch. Three full power buses. Sure they are all generator backed (with aggressive testing cycles) UPS backed, yada, yada as with other centers I use. But they also insist on customers taking power from two buses so if, in spite of generator and UPS protection, a power bus fails you will still have power. And since they only run at 2/3 power/bus, they can reroute power and continue providing dual redundant power if one bus is down.

      --

      ~~~~~~~
      "You are not remembered for doing what is expected of you." - Atul Chitnis
    31. Re:Just off the top of my head by Anonymous Coward · · Score: 0

      haha

    32. Re:Just off the top of my head by georgewilliamherbert · · Score: 1

      Let's be clear - if you have hot-aisle cold-aisle with proper plenum separation, i.e. either walls or curtains blocking airflow from cold to hot other than via the racks and systems - then you can run all the air from the ceiling fine and it doesn't matter.

      If you do not do that, you have to pump up the cold air pressure to jet it down so that it goes down into the cold aisles to floor level, and does not get sucked right back up into the return vents in the hot aisles in a loop up near the ceiling.

      Once you pump it up with enough pressure to do that, you're at the same energy usage you would have had running it down into a raised floor plenum.

      IF your datacenter has those hot aisle / cold aisle plenums with physical separation so it doesn't short circuit, concrete floor is fine. I've seen exactly two places that put them in, one of which screwed it up and was in fact in worse shape than before.

      IF you just spray it down from above without separation, and you don't have enough cold air pressure, you are losing all the cooling efficiency circulating air without running it down to/through servers. Convection will make the hot aisle air rise, but not fast enough to win entirely.

      Raised floor is not perfect. But it does have the advantage that if you cold aisle hot aisle with overhead exhaust, you can be reasonably sure the cold is done right just by managing the ventilated floor squares right and doing a little measuring. It is MUCH MUCH SIMPLER to make work with reasonable effectiveness and efficiency and much less likely to be grossly wrong.

      If someone says "My datacenter floor is concrete for energy efficiency" and does not immediately then describe the curtains or plenum separations for hot/cold aisle, they are bullshitting you. They heard that it's better but don't know what will kill you in doing it that way, and they are probably going to get it wrong, orat least not get it right enough to save energy. If you find the right people and they're doing the proper plenum separation then go for it. If not, don't. The other people should use raised floors until the knowledge on plenum separation soaks out into the industry enough that they can do it right.

    33. Re:Just off the top of my head by GaryOlson · · Score: 1

      SCREENWASH!
      ROFLMAO!

      And an infinite budget...

      --
      Every mans' island needs an ocean; choose your ocean carefully.
    34. Re:Just off the top of my head by rijrunner · · Score: 1

          Do they have remote console capability?

          Onsite or offsite support?

          Level of skills of onsite support.

          How many people are onsite during offhours?

          Is your equipment secure?

          Can you come out and service your equipment? Do you have to be escorted?

          How quickly can equipment go from being ordered to being deployed?

          What are their change procedures? Are their change windows when all work needs to be done?

          Do they have phones in the server farm? If so, how close to your servers? Cell phones?

          Are they operating their systems headless? Crash cart? HMC? Management consoles?

      The physical stuff is fairly easy to evaluate. Generally, you can get a feel for that in about 60 seconds. But, the questions above can turn a 20 minute glitch into hours of outage. Having a remote console or RSA ports connected to the network can save someone from having to go to the farm.

      Your own architecture will also define a lot of the acceptable answers. If you are running large systems with a lot of remote access via management consoles, you are not as concerned about a lot of these. If you are running standalone boxes, some of these will be brutal if they are not in place.

      If it takes a week to schedule one of your people to come onto site and be escorted to your machine, then you have to leave because their escort has something more important they have to do.. you have the wrong farm.

      And no.. that's not hypothetical.. I was stuck in that situation too many times with one company..

    35. Re:Just off the top of my head by Gorobei · · Score: 1

      The typical (though changing) datacenter has mixed hot and cold air - typically cold air pumped up from the bottom (?!? kind of fighting nature, there) then allowed to rise into the ceiling.

      It's not fighting nature, it's using it right. If you push cold air in from the top, it warms up, and wants to go back up. Push it in from the bottom, and it does its natural thing and wants to exit via your top exhaust.

    36. Re:Just off the top of my head by padyer · · Score: 1

      +1

    37. Re:Just off the top of my head by kilodelta · · Score: 1

      Also - data centers do move. I know, been through a couple myself. Make sure the movers are bonded, check references for the movers, and insure both the equipment and costs to recover lost data.

      One move was government systems. One of the horror stories I heard back when checking with other state agencies was that in one case movers dropped an entire rack of servers, destroying all.

      Our moving company un-racked every server, then wrapped in padding and blanket and placed in a stable rolling cart. Only 3 servers per cart.

      Move went off without a hitch. However the facility we moved into was still under construction and we had a brand new fully populated HP4108 switch that got fried because the electricians shorted the power supply while running CATV lines. Apparently a piece of copper fell in and the electrician heard a snap and we noticed our network going dark.

      Luckily I could get some spare switches from another state agency. And the building owner was good, he paid the $1,500 plus $150 in shipping charges to get us a new power supply for the HP switch the next day. We also chipped in $1,500 to get the redundant power supply.

      Disaster recovery is clutch. We had a web server crash hard that didn't have solid backups or documentation. So we put policies in place to document and backup everything. Ended up using an rsnapshot server for the purpose and that made life so easy.

    38. Re:Just off the top of my head by Critical+Facilities · · Score: 1

      But please, continue to refute my statement with clear, unsupported, single-word denials. They carry so much weight in an argument.

      Yeah, you're probably right. I mean, no one is putting in raised floor environments anymore. I don't know what I was thinking.

      Quote all you want. I run an enterprise data center, and I can tell you that raised floor is certainly NOT dead.

    39. Re:Just off the top of my head by coaxial · · Score: 0

      Man Traps? Nightly security audits? What type of stuff is on those servers? It seems like overkill for a typical business setup.

    40. Re:Just off the top of my head by Anonymous Coward · · Score: 0

      add dual power feeds to each rack, from different power grids.

    41. Re:Just off the top of my head by aaarrrgggh · · Score: 1

      Even if you are going co-lo, you are best off hiring a pro to help you. (Disclaimer: I am a pro.) The checklist approach is a good starting point, but far too often you will miss site-specific issues.

      How old are the UPS batteries? Where are the preventative maintenance logs? What happens if the facility is expanded? Are there any single points of failure you need to understand? Is it 6N/5 or N+1? Are there lease terms that you need to be careful with?

      We have done big deals with large customers, and despite having a good internal team, good legal experts, and it is always amazing what little things come up that are really critical to addressing properly to protect liability.

      Purchasing a facility is often easier; you either know the facility is a write-off, has some level of residual value, or hope that it will actually be viable off the blocks.

      The hardest decisions are from M you always have to be careful for deferred maintenance and disappearing staff.

    42. Re:Just off the top of my head by atrus · · Score: 1

      You want contained hot aisles or contained cold aisles to maintain maximum efficiency. You want managed airflow.

      Its perfectly ok for the hot aisle to be at 100+F. Its also perfectly ok for the cold aisle to be at the mid 70s, as long as there is no stratification or leakage (top of the rack should be within limits). What you want to see is offline CRAHs or VFDs installed in the CRAHs throttling back their airflow.

      I work at a company which specializes in monitoring and helping customers improve their capacity and energy use. By placing air where it needs to go you can both cut costs and improve capacity. Sadly, very very few datacenters are run with the efficiency and managed airflow in mind (people even put perforated tiles in the hot aisle!). If you have containment, then you're 95% of the way there. Another more modern option is liquid cooled racks, or in-row cooling (APC Pods and the like).

    43. Re:Just off the top of my head by Anonymous Coward · · Score: 0

      no such thing as overkill in terms of security when dealing with high-end clients (read govt.) in a datacenter

    44. Re:Just off the top of my head by zonker · · Score: 0

      One thing I learned about from the EDS failure back in 1999 or 2000 (can't remember exactly) but there was a series of bad storms in Georgia that flooded their main data center. It led to a nationwide failure of ATM machines that lasted several days. Their data center was modern and had everything it should have had. Except it wasn't above the flood zone. BIG mistake and it cost the bank I worked IT for a lot of grief.

      So having an idea of the physical location can be just as important as things like how many 9s their service record shows.

    45. Re:Just off the top of my head by Anonymous Coward · · Score: 0

      Cooling capacity is measaured in kW

      Cooling capacity is still measured in 'tons' once you look up from the design specs and start talking to the manufacturers and integrators. Yes, it is retarded.

    46. Re:Just off the top of my head by JWSmythe · · Score: 1

          That's another whole story. :) Don't make me tell you about the Catalyst 5000 switch I bought on eBay, just to find it was still configured for the classified network it was on.

      --
      Serious? Seriousness is well above my pay grade.
    47. Re:Just off the top of my head by JWSmythe · · Score: 1

      I'm really happy to see when people know what they're talking about. :)

      At one datacenter, I worked closely with the site engineer Our cage was fairly stuffed, and he used ours as an example to newbie customers on how to do some things. The center of our cage was our cold "aisle". The outside was our hot side (just two rows of servers). He'd bring his infrared temperature sensor along, and point out the back of our rack was only about 10 degrees hotter than the front, because we did account for good air flow. They had a much more robust customer (several thousand machines) who followed our lead as they were populating their cages, and they never had heat problems. I could actually walk around their area, and barely feel the temperature difference.

      The bad site I mentioned, I felt the heat walking through areas that shouldn't have even been warm.

      If anyone wants to know the site, email me. I won't post it. They were bad though. They were pretty bad about other things too. Like, "I'm bound by contract not to say who's in that space, but they are the huge search engine, who's name rhymes with zoogle". Come on guys, how am I suppose to trust that you won't disclose anything about my company now?

      Needless to say, we never moved in there.

      --
      Serious? Seriousness is well above my pay grade.
    48. Re:Just off the top of my head by Glendale2x · · Score: 1

      A 12" slab with mesh (what mine is) does have a higher load bearing capacity that a raised floor, though. Equipment weight and load distribution is completely a non issue.

      --
      this is my sig
    49. Re:Just off the top of my head by Fulcrum+of+Evil · · Score: 1

      Left this part out: (Not a DC admin)

      --
      "We returned the General to El Salvador, or maybe Guatemala, it's difficult to tell from 10,000 feet"
    50. Re:Just off the top of my head by Critical+Facilities · · Score: 1

      While that's true, it's not really too much of an issue. The really, really heavy equipment (CRAC Units, PDUs, RDCs, etc) typically have their own stands, which are placed directly on the concrete slab below the raised floor. The advantages in flexibility with regard to airflow and cable management that exist with a raised floor environment are numerous.

      Mind you, I'm not suggesting that one cannot have a successful, efficient data center without a raised floor, I am merely responding to an earlier poster who implied that non-raised floor data centers were somehow more efficient, which I maintain, is incorrect.

    51. Re:Just off the top of my head by Sandbags · · Score: 1

      I had to be locked in one over an entire weekend VOLUNTARILY.

      Major server overhauls, were going to take all weekend to do full backups, replace drive cages, rebuild, and then restore.

      The client operated the top 3 floors of a 15 story building, but because of the design, had I not been locked in a secure area (datacenter), I would have had access to several other floors I was not supposed to be on that were wide open from the elevator (no loched doors). Security locked up the whole place Fridays at 8PM and did not come back until monday at 5AM, and people were not normally left to be in the building without security watching elevator cameras...

      The server room thankfully had its own integrated bathroom, and I brought in a cooler, a portable TV, a game console, and a folding cot. I spent the weekend in there alone. Made a SHITLOAD of overtime doing it (some of it double and a half time based on our billing methodology). Will NEVER do something like that again willingly...

      --
      There is no contest in life for which the unprepared have the advantage.
    52. Re:Just off the top of my head by Sandbags · · Score: 1

      2 of our 6 datacenters are built using the hot/cold isle method. 2 others are in a migration to a newwer design that still uses hot/cold, but it;s all intra-rack. The datacewnter itself will be a nice 76 degrees, but inside the racks will be 65 or lower. Cold moisture free air is pumped in from the top front of the rack. Servers pull it through to the back and hot air is pulled up and out. The racks are sealed for both airflow and noise reduction. Each rack chassis is about 6-8" deeper than a normal rack, but because of efficincy we can place about 1/3 more racks in a datacenter and still have ample cooling. We currently have 5 air handlers in each large datacenter, and 3 each in the smaller ones. Any one can be offline completely assuming we're at max capacity, honestly any 2 likely could be off as our datacenters, though designed to be full, are rarely much more than half populated, as it would only be full during a period when another datacenter is being rebuolt (every 3-4 years or so on our pace).

      --
      There is no contest in life for which the unprepared have the advantage.
    53. Re:Just off the top of my head by Col.+Panic · · Score: 1

      also water detection. it's pretty handy to know as soon as possible when your cooling system is leaking under the raised floor.

    54. Re:Just off the top of my head by Sandbags · · Score: 1

      in a dul row arrangement, cold air will never "warm up" on it's row. It has to be pulled through a server from the cold side to the hot side where it then rises.

      2 of our datacenters are being remodled to go a step further, and not cool the space between racks, only what's in the, in a similar design (sealed racks). so the additional several thousand square feet of the datacenter need only be maintained at a nice comfortable room temp.

      --
      There is no contest in life for which the unprepared have the advantage.
    55. Re:Just off the top of my head by Sandbags · · Score: 1

      Medical data mostly, but naturally that also includes billing records, credit cards and bank transaction data, and lots of other information that should not be leaked. Of course, the $150M in equipment something to be concerned over as well, especially when you have a few hundred employees and contractors with datacenter access on a regular basis.

      We're not a "typical business" because of the data, and also because of who owns many of the servers in our datacenters (only about 2/3rds actually belong to us directly, and half of those go with a contract if we loose it to a competitor). We have numerous different government (state and federal) security standard to adhere to. We go through an external security audit from one group or another every few weeks ensuring we're sticking to the standards for their servers and their data.

      --
      There is no contest in life for which the unprepared have the advantage.
    56. Re:Just off the top of my head by rauusp · · Score: 1

      KW available to each rack are essential. Colocation services are ridiculous at the moment, you get a 42 U rack and can only stick 3 or 4 2U machines because of power restrictions.

    57. Re:Just off the top of my head by Critical+Facilities · · Score: 1
      I'm definitely not arguing with you here, and I agree with all the supplemental points you added. But, in the interest of clarifying my points:

      Power quality: never seen a big datacenter without a Liebert, or at least UPS in every rack. Power does not have the be contitioned except between the UPS and the machines/devices. A whole data center power conditioner is often more efficient, but unnecessary for the little guys. either way - check.

      I would argue that incoming Power Quality from utility is still a key factor worth looking into. While it's true that your UPS(s) are going to correct any power quality problems, this also means that your UPS(s) have to work harder to correct the problem(s), and particularly in the case of low Power Factor, you will get less actual power available (as you will begin to encounter current limitations on the "line" side of your UPS(s) before you've reached your designed capacity on the load side.

      Also, when I refer to Power Quality, I'm also referring to the frequency of power outages, spikes, and brown outs on any of your utility feeds. This will be a good indication of how much your UPS is being hit, and how much risk your equipment is potentially being exposed to. In the case of a conventional UPS with batteries, when there are frequent brown outs or spikes, it is often advisable to put a Flywheel UPS in series before the Standard UPS to absorb the small "hits" in order to extend the life of your batteries.

      - Fire suppression is usually part of your building codes, and a given, as is the routine checks (at least anually) by law.

      I would agree, except that there are tests that are not always required by law that should be performed. In the case of Pre-Action Sprinklers, MIC Testing is something that should absolutely be looked at. Also, in the case of any Gas Suppression Systems (i.e. FM-200), routine checks for any potential leaks could prevent a very, very expensive discharge.

    58. Re:Just off the top of my head by Anonymous Coward · · Score: 0

      Raised floor height - high enough to hide in when your boss comes looking for you
      Cooling Capacity - should take 'er all the way down to about 36f to keep your beer cold
      Security - You should be the ONLY one who can get in or out of everything. If there is a key or a card (except yours) that gets into everything, start changing the locks.
      Power Quality - this is why you need a 40 inch Plasma TV in there. If that flickers you need some power filters.

      Assess other criteria in the same vein.

    59. Re:Just off the top of my head by Evolt's+RonL. · · Score: 1

      Liquid cooled server racks ... Hey! ... Kerosene is a liquid, isn't it?

    60. Re:Just off the top of my head by guile*fr · · Score: 1

      like storing beers and the occasional clumsy tech.

    61. Re:Just off the top of my head by Anonymous Coward · · Score: 0

      8 feet?! What beast lurks beneath that? God forbid you move a tile off and someone falls _in_.

    62. Re:Just off the top of my head by SuperQ · · Score: 1

      Wow really? That's the best examples of datacenters you can find? Not a single mention of PUE at all. Lots of fancy talk and equipment but no real numbers.

      Cisco's datacenter consumes 10MW. If it has a PUE of 1.50 (good for that kind of design) that's only about 6.6MW for racks.

      If this were a Google datacenter with a PUE of 1.22 that's 8.2MW for racks.

      If cisco can maintain a PUE of 1.5 for that design, they're wasting 1.6MW of power. That's about 13670MWh per year or at $0.08/KWh is about 1.1 millon dollars a year.

    63. Re:Just off the top of my head by Critical+Facilities · · Score: 1

      Wow really? That's the best examples of datacenters you can find? Not a single mention of PUE at all. Lots of fancy talk and equipment but no real numbers.

      No, those are the first few examples I could quickly find as examples of modern data centers being designed specifically utilizing a raised floor environment. Bear in mind that I was responding to an individual who seemed to be under the delusion that having a raised floor environment somehow relegated a data center to a "less efficient" status, and I was using recent examples to illustrate his error.

      You're correct that the lower the PUE for a data center, the more "throughput" (for lack of a better term) one gets for the energy being consumed. That said (and admitting that I am a fan of Google and their innovations in the field), I, like many others, am quite suspicious of their reported 1.22 PUE. I have yet to find any actual data to illustrate and corroborate how they have calculated this. If they are able to maintain that type of PUE in each fully loaded data center, that's pretty impressive. I realize that they've reported that they're able to do that, but all the stories that I've read seem to report merely that they've achieved this, and no "proof" to back it up. I suspect that we're seeing average or weighted average numbers which are incorporating some less than fully loaded facilities, which is dragging the average for the others down.

      Admittedly, this is just my opinion.

    64. Re:Just off the top of my head by ZerdZerd · · Score: 1

      How do you get a guided tour at a data center?

      --
      I'm not insane! My mother had me tested.
    65. Re:Just off the top of my head by SuperQ · · Score: 1

      Google publishes how they calculate it. It's pretty simple. here is the link. They also post updates quarterly with each datacenter listed separately, although not which one is located where.

      Other companies (Yahoo? MS? I forget) are doing similar things with PUE nearing 1.1 by only running the datacenter for peak capacity handling and the just shut it down when it's too hot out to run evaporative cooling.

      Either way, using raised floor for cold air handling is not the way you get better than 1.5 PEU.

    66. Re:Just off the top of my head by Critical+Facilities · · Score: 1
      From the article you've linked to:

      Our numbers include all power-consuming items in the facility, with one single exception (power used by the office area).....the facilities themselves are located in different climates, which influences the PUE performance....PUE values are impacted by seasonal weather patterns, and thus the PUE during cooler quarters tends to be lower than in warmer ones

      This explains some of the lower numbers that they're achieving. While I do not disagree with the methodology, and I do agree with Google that this type of measurement is truly more in line with the spirit of PUE measurement, most facilities do not have the type of sub-metering in place to exclude "office space" electrical loads from the overall measurement. I would argue that if one could make the calculations as Google is doing (measuring exclusively the data center, excluding all other "office" type loads, and including data centers in cooler geographic/seasonal locations in the averages), we would see a trend of lower PUEs being reported at other facilities.

      Now don't get me wrong here, I am a fan of Google. I think they're very smart, and I like a lot of their innovations. I interviewed with them extensively, and almost chose to work for them running one of their data centers, believe it or not. With that said, I still disagree with your assumption that "using raised floor for cold air handling is not the way you get better than 1.5 PEU (sic)". I am arguing that there are very effective methods of impacting PUE that incorporate raised floor environments, like hot aisle/cold aisle containment. I also argue that things like water or air side economizers and ultrasonic humidification systems (as opposed to infrared or other heat generating humidification methods) have a very large impact on PUE. In short, you and I agree that Google is doing a great job at achieving and driving lower and lower PUE, we just disagree that raised flooring is an obstacle to that.

      In fact, one of the articles that is linked in the footnotes of the article you linked to talks pretty specifically about utilizing raised flooring and various ways to improve upon the airflow by maximizing some of the facets of raised floor system. I don't mean you any disrespect, and I am not trying to incite an argument/flame war. Rather, I'm merely trying to dispel an idea of raised floor environments somehow being inferior.

    67. Re:Just off the top of my head by JWSmythe · · Score: 1

          Work for a company with a money and an intention of moving into a DC. Sales folks will usually get you the tour. They don't always just offer it, because lots of people who are just negotiating the contract wouldn't know the difference between a datacenter and a broom closet. It depends on the facility though. For example, Rackspace only does managed hosting. They don't let anyone on the floor. I assume they aren't running out of a broom closet, but even though I've been a customer in a couple of their facilities. I was told customers were never allowed in.

      --
      Serious? Seriousness is well above my pay grade.
  2. first by electrosoccertux · · Score: 0, Offtopic

    you evaluate it by how many cups for frosty piss it can...
    wait...
    in soviet russia, data centers evaluate you!!!!
    Yeah there we go.

  3. History by micksam7 · · Score: 2, Insightful

    Look at a datacenter's history [recent and past], outages, maintenance issues, customer support, management and etc, in conjunction with their listed redundancies and capacities.

    Just because they have two electrics going to each server, doesn't mean a random maintenance tech will flip the wrong switch. :)

    1. Re:History by marbike · · Score: 1

      Nor does it mean that when they lose a leg of power that it will cut over nice and neatly. Got bit by that a while back. Their power bump put the air handlers offline. Ten minutes later all of our servers went into thermal shutdown.

      --
      it is better to light a flame thrower than curse the darkness. -Terry Pratchett Men at Arms
    2. Re:History by Nefarious+Wheel · · Score: 1

      And of course, don't forget the simple visual inspection. Sometimes the cabling infrastructure may not quite be up to spec.

      --
      Do not mock my vision of impractical footwear
  4. attack it by Anonymous Coward · · Score: 2, Insightful

    set it on fire, throw floods at it, generate tornados, then top it off with a nice earthquake.

    1. Re:attack it by Captain+Splendid · · Score: 1

      Coming in 2010 - Will Wright's SimDataCenter!

      --
      Linux, you magnificent bastard, I read the fucking manual!
    2. Re:attack it by AndrewNeo · · Score: 1

      I want to play that now, darnit!

  5. past outages by Anonymous Coward · · Score: 0

    Make sure you ask about any past outages and how they were handled, I have seen data centre power that has failed 4 times in 1 year due to the same problem that was only found on the fourth outage.

  6. Re:Get this out of the way by Kratisto · · Score: 1

    Let's not forget about watts per library of congress.

    --
    Conscience is the inner voice which warns us that someone may be looking.
  7. i ran a junky data center by digitalsushi · · Score: 4, Informative

    I ran a data center long, long ago. My sales guy knew it wasn't going to pan out and threw me to the wolves. He asked me to start the tour, and then he took a long lunch to miss it.

    The guys I gave the tour to seemed very intelligent. They only spent about 60 seconds on our data center. The instant they saw the carpet, their eyebrows were up. When I didn't lie to them that there was no diesel generator on the other side of the (secretly dead) batteries, they did exactly what they should have and stormed out without saying thanks.

    --
    slashdot: where everyone yells sarcastic metaphors to themselves to understand the issue
    1. Re:i ran a junky data center by syousef · · Score: 0

      I ran a data center long, long ago...... The instant they saw the carpet, their eyebrows were up. When I didn't lie to them that there was no diesel generator on the other side of the (secretly dead) batteries, they did exactly what they should have and stormed out without saying thanks

      Sounds like you ran it out of your mother's attic. (A basement wouldn't have had carpet).

      --
      These posts express my own personal views, not those of my employer
    2. Re:i ran a junky data center by Anonymous Coward · · Score: 0

      (A basement wouldn't have had carpet).

      Mine does.

      It is over concrete, but it's carpeted.

    3. Re:i ran a junky data center by Nefarious+Wheel · · Score: 3, Funny

      A smattering of basic physics helps.

      Long ago in a distribution centre a far far away - well, east SF bay, anyway - we had a custom mini doing a bit of work for a major retail store chain's logistics business. In the warehouse they built a little room for the mini upstairs, everything cheap but per spec, they insisted. They used one of their domestic air conditioners for the cooling, as it had the right thermal rating to match the heat dissipation we required for our gear. Cool, we said - no problem, cheap is ok as long as it's specced correctly.

      It wasn't long before we had a service call for a hardware failure. Sent the engineer out, and it was about 110 in the computer room. They'd installed the air intake and air outflow of the air conditioner in the same tiny room.

      --
      Do not mock my vision of impractical footwear
    4. Re:i ran a junky data center by NervousWreck · · Score: 1

      Carpet?! Ouch. My sympathies. Were there also space heaters?

      --
      I do not have a sig. You are hallucinating.
    5. Re:i ran a junky data center by Anonymous Coward · · Score: 3, Funny

      I think "data center carpet" should be a new slashdot meme. I can not stop laughing at how ridiculous that "data center" must have looked with that carpet. Please tell me that it was the baby poo green shag carpet from the 70's. That would really make it feature complete.

    6. Re:i ran a junky data center by AndrewNeo · · Score: 1

      I think you have basement and attic backwards..

    7. Re:i ran a junky data center by Zerth · · Score: 1

      They'd installed the air intake and air outflow of the air conditioner in the same tiny room.

      Wow... I bet they sometimes leave the fridge open to cool the break room.

    8. Re:i ran a junky data center by upuv · · Score: 1

      I have a phrase I use.

      "You can never out smart an idiot!"

      You think you have ever angle covered. You are proud of how tight the whole design is. Then along comes some moron and BOOOMMMMM. They defeat your planning in 1 second.

      Happens all the time.

      I've had a lot of managers in companies look at me when I say that. First off the idiots get very offended. The smart ones get it. The phrase acts like pre-filter before any work is actually done.

      As a result over the years I have learned that design and implementation has to have certain attributes.
      - flexibility. We will have to come back and alter/fix something.
      - redundancy. Their has to be more than 1 method of doing something. In your case a window and a fan as my backup second option.
      - slack. Things never fit exactly into the specifications. Whether it be a measurement of distance. A power consumption claim. or a throughput estimate. Always add more during the build. It's cheaper.

      It's the arrogant that think it works out of the box first time.

  8. Additional Questions by Astrobirdr · · Score: 3, Insightful

    I'd also ask:

    Number of years in business.
    Involvement of the owner in the current business.
    Number of years the current owner has been in this business.
    Also do a check with the Better Business Bureau to see what, if any, complaints had been filed.

    And, as always, Google is your friend -- definitely do a search for the business you are considering along with the word(s) problem, issue, complaint, praise, etc!

    1. Re:Additional Questions by djweis · · Score: 1

      I'm not sure BBB involvement is relevant for a data center. It's not an auto repair shop with thousands of customers per year. Searching the online court records would be a more appropriate resource.

  9. Datacenter Archaeology by jpvlsmv · · Score: 2, Funny

    Pull floor tiles and compare the amount of obsolete technology-- Thicknet cables, VAX cluster interconnects, water chiller hookups, FDDI cables, etc. with the amount of space remaining.

    Anything less than 4 inches of obsolete crud isn't worth excavating. Leave it a few more years.

    --Joe

  10. Security by Anonymous Coward · · Score: 1, Funny

    I co-locate at a data center in Alberta. It is in the basement of a high-rise building. Because of this there is much traffice in/out of the building. The main doors within the building leading to the datacenter itself can be opened with a credit card, or even a set of keys (um, any key). This poses a security risk. Even though you'd need to know exactly where to go and when (so as to not bump into people working there), it is still possible to get what you're after realtively simple with no alarms. They do have cameras though, so wear a mask - and since this is Alberta, no one would question the mask.

    I picked up one of my servers a couple days ago, and they didn't ask for ID either. I could have been ANYONE.

    1. Re:Security by Anonymous Coward · · Score: 0

      Care to name-drop?

    2. Re:Security by TheLink · · Score: 1

      Why? Going there to pick up your server?

      --
  11. I evaluate it dusly... by Anonymous Coward · · Score: 0

    ... If my direct manager evaluates it as good, then im good. If my direct supervisor evaluates it as fucked, then im fucked.

  12. Word of mouth by tomhudson · · Score: 4, Insightful

    Find someone you trust who's already a customer. Word of mouth beats any number of white papers or studies or guarantees.

  13. Some important questions: by swordgeek · · Score: 2, Informative

    I'm assuming this is evaluating for co-location purposes. Here are some things I'd ask.

    1) How quickly can I get a new server deployed into it? How do I do it?
    2) Can I get a tour? Now? (Note that this not only lets you see the data centre, but also will give you an idea of security. Look for procedures on getting in, notice if they ask you to sign a release form, etc.)
    3) How close to capacity are you? (The answer should include space, floor weight, power, cooling, and network. If it doesn't, why not?)
    4) What are your racking/networking/cabling standards? (They should have some, at least where you connect to them, but they shouldn't be onerous).
    5) How many people manage the data centre? You don't want to be one car accident away from loss of access or service.
    6) How about power management? Is the centre on a UPS, redundant UPSes, or nothing? Can you get charts of the power going to the servers? Can you get DC for telecom servers, or only AC? Is it on a generator for long-term outages? (Note that you may not need this--in which case you shouldn't pay for it. Alternatively, if you need it, make sure it's there!)
    7) Is it manned 24/7? (Ditto!)

    If you can, ask them to pull a tile so you can see under the raised floor. Underfloor cabling (and suspended ceiling cabling for that matter) should be neat, tied, and labelled. Dead cables should be pulled, not left to rot. There has to be sufficient clearance for unrestricted airflow. Cages are better than lying on the floor.

    Most of what makes a good data centre comes down to organization. If it's a rats nest, then even if there's one guy who knows "everything," it will be less reliable, less consistent, and less predictable. Procedures should be written down, printed, filed in labeled binders, and regularly updated. (Note: Online copies should be canonical, but also needs to be accessible offline when shit --> fan.)

    Fire suppressant mechanisms (wet vs. dry, live pipes, etc.) need to be considered, as does emergency lighting. If the operators need to start digging around for a flashlight to read what they should be doing, then things aren't happening the way they should.

    Be picky. If they're leasing space to you, then their data centre design and maintenance is their BUSINESS, and they had better get it right! Look for a neat, well-organized, well-documented, well-panned data centre. Also make sure that it fits your needs.

    --

    "People who do stupid things with hazardous materials often die." -- Jim Davidson on alt.folklore.urban
    1. Re:Some important questions: by Red+Flayer · · Score: 3, Interesting

      If you can, ask them to pull a tile so you can see under the raised floor. Underfloor cabling (and suspended ceiling cabling for that matter) should be neat, tied, and labelled. Dead cables should be pulled, not left to rot. There has to be sufficient clearance for unrestricted airflow. Cages are better than lying on the floor.

      Just want to add... Don't let them pick the tile. They probably get this request frequently enough that they have a "show" tile or two if they are a shoddy organization. Pick one on your tour, as an offhand request that you had "forgotten" until then. If they try to steer you to a specific tile, that tells you they have something to hide, and you need to question everything else they've shown you samples of.

      [paranoid and loving it]

      --
      "Trolls they were, but filled with the evil will of their master: a fell race..." -- J.R.R. Tolkien on Olog-hai
    2. Re:Some important questions: by vlm · · Score: 3, Interesting

      Just want to add... Don't let them pick the tile. They probably get this request frequently enough that they have a "show" tile or two if they are a shoddy organization.

      If you pull this stunt, please understand that a techs hidden stockpile of magazines and canned soda does not necessarily indicate a shoddy organization, it merely means they have employees that like reading certain magazines for the interviews, and prefer to store their drinks in a nice clean spot underneath the chiller rather than the proverbially filthy employee refrigerator. On the good side this is a strong indication they don't have an under the floor rodent infestation.

      Strangest thing I ever found under the floor was a vast amount of one employees (clean) clothing. He was kind of stuck in the process of moving and needed a temporary place to stash stuff. Apparently no one found it unusual that he was hauling bags of clothing in and out.

      --
      "Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
    3. Re:Some important questions: by Anonymous Coward · · Score: 0

      "Dead cables should be pulled, not left to rot"

      After 25 years in telecoms I disagree.
      There are far more troubles caused by pulling out old cables then there are in just leaving them there.
      To remove a cable without causing damage to those around it is very hard work and most installers cheat and just pull the cable out.
      That results in damaged cables around the original cable.
      If you are lucky - a failure will be immediate and obvious. If not, have fun looking.
      I remember one really weird temperature related fault that took ages just to figure out it was a cable fault.

  14. What are you evaluating? by chris.knowles · · Score: 4, Insightful

    There are basically 3 perspectives from which to evaluate the Datacenter. They're pretty well universal to any IT eval. People, Process and Technology. The datacenter facility itself is only one piece of the puzzle (Facility = Technology, which only accounts for a fraction of the total cost of operating a Datacenter). There are also the people running the datacenter and how they are organized and interact with the technology, one another, and their customers (internal and external). From a people/process standpoint, if you want to give a general "score" to them, you can assess them against the SLM maturity scale. (Read about the Gartner Maturity Model for Infrastructure and Operations) Evaluating a datacenter is going to be a balance between the cost of operating the datacenter and the level of service you require from said datacenter. There really isn't enough information in the question to give you a good answer. Are you looking at evaluating the acquisition of a datacenter to grow into, are you looking for a managed services DC to host your gear with operational support? Are you looking for rack space with pipe and power? If you give more details to your inquiry, I'm sure the community can provide you with some great answers.

  15. Duh by antifoidulus · · Score: 0

    based on the Data Center Chicks of course!

    1. Re:Duh by AndrewNeo · · Score: 1

      Must.. buy hosting... from GoDaddy!

  16. Do not jump in with both feet by Jailbrekr · · Score: 3, Informative

    Regardless of how well they are decked out, always start with a "pilot project". Start small for a short period to evaluate real world performance of both their equipment and their tech support. We currently have a pilot project in place to evaluate a datacentre for outsourcing our compute requirements. We have learned that while they have exceptionally good equipment in place, their responsiveness and ability to provision is highly questionable.

    --
    Feed the need: Digitaladdiction.net
    1. Re:Do not jump in with both feet by rijrunner · · Score: 1

      Honestly...

      Have you *ever* seen a pilot project where they had not already made the decision to deploy regardless of the outcome?

      It always plays out the same way..

      Client admin: we had problem A, Problem B, Problem C
      Sales dude: Hey.. we worked out the bugs...
      Client CIO: Cool. Let's move the rest in now that all the problems are fixed.

      Honestly.. pilot programs are supposed to have problems. You can't get a good feel for the situation because almost invariably you are working with project support and not steady state support in a large farm. They will throw resources to fix major problems when they know there is a sale on the line. Its *always* questionable, even if you have great support during the pilot. If you are dealing with their steady state support during the pilot, you might actually getting a better deal than elsewhere.

      Seriously.. pilot programs are *never* representative. You either have more resources available or more technical issues to be resolved than you will have in steady state.

    2. Re:Do not jump in with both feet by TheRaven64 · · Score: 1

      If you have the resources, start pilot projects in a few data centres and then go ahead on one or two after you've evaluated them as a customer for a bit. This works best if you don't tell them that's the plan. See how they treat you as a small customer; they'll treat you at least that well as a large one.

      --
      I am TheRaven on Soylent News
  17. Simple... by Anonymous Coward · · Score: 0

    Read their disaster preparedness plan. If you can get through it without your BS alarm ringing off the wall or laughing hysterically, there is hope. You do have a disaster preparedness plan, right?

  18. What do you need? by Tdawgless · · Score: 2, Insightful

    What does your company _NEED_? How much bandwidth do you need? What kind of servers do you need? Are you looking for Co-Lo or Dedicated? If you're doing Co-Lo, how much power and space do you need? If you're doing dedicated, do you need managed or unmanaged? PCI compliance? HIPAA compliance? Do you want to pay for certain redundancies? Do you need an Uptime Institute Tier certified facility? I could go on and on. The one thing that you need consistently is good customer service. The rest depends on what you need. Full Disclosure: I work for one of the biggest privately held dedicated hosting companies on the planet.

    1. Re:What do you need? by Anonymous Coward · · Score: 0

      *hint* The Planet

  19. Vending machines. by Kenja · · Score: 2, Insightful

    Since the odds are I'm going to be spending the night there at some point, good vending machines or a cafeteria are a must.

    --

    "Have you ever thought about just turning off the TV, sitting down with your kids, and hitting them?"
  20. Real Simple by MrKaos · · Score: 1

    Power from the ceiling, data under the floor.

    The reason is data centre floods don't occur very often but when they do the d.c can tolerate the data cable being in water but when the power gets in contact with water circuit breakers trip and they don't work again until they are dry.

    I encountered it when the AC water feed burst and co-incidentally the drain for the data centre had been blocked. If your power and data are through the floor then I would suggest that you invest in a good wet and dry vacuum cleaner. I do have other suggestions but this seems such a basic thing to me.

    --
    My ism, it's full of beliefs.
    1. Re:Real Simple by jackb_guppy · · Score: 1

      IT was data room not a data center and it was 25yrs ago...

      On the third floor was data room with computer and phone switch next too the main A/C flume going from basement to 20flr (walking closet without floor or ceiling. The external auditors came though and wrote up the location that there was no raised floor in case of flooding.

      Site's response: HA HA HA - Flood - 3rd floor?

      Next week the water main at the 5th floor brpke and came down the A/C flume and flooded the third floor in 2 feet of water!

      Opps

    2. Re:Real Simple by MrKaos · · Score: 1

      Next week the water main at the 5th floor brpke and came down the A/C flume and flooded the third floor in 2 feet of water!

      Exactly, mine was on the fifth floor!!!

      --
      My ism, it's full of beliefs.
    3. Re:Real Simple by Anonymous Coward · · Score: 0

      I was in a single story building, but the dehumidifier got clogged up and a river ran through it... it being the underfloor cabling. The amazing thing was we were able to shutdown cleanly after I noticed.

  21. My Top Three by Archangel+Michael · · Score: 0, Troll

    1) Hookers
    2) Beer
    3) Illicit drugs

    Seriously, my top three are as follows ....

    1) Bandwidth Available / Oversubscription rate
    2) Geographically different alternative location.
    3) Disaster Planning directives.

    --
    Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  22. Organization by jroc242 · · Score: 1

    After redundant power, adequate cooling, 24/7 ops...etc. Neatness is the most important thing. Making each rack very neat with redundant power bars and individual patch panels is key for the long term. Its when you allow cables to run under the floor and between racks that things get out of control. We have an old timer that will yell at anyone who does anything messy and its works out very well.

  23. Re:Get this out of the way by Captain+Splendid · · Score: 0

    Rods/hogsheads...

    sleepy now

    --
    Linux, you magnificent bastard, I read the fucking manual!
  24. Don't forget the non-technical bits by petes_PoV · · Score: 2, Interesting

    Such as street access. Is there more than one way in, if the access road was closed off (police incident, subsidence, civil unrest - depending where it's sited), what would happen. Could staff get to work, or leave for home?
    Ease of recruiting / retaining sufficiently qualified staff in the locale, or persuading your to commute or relocate
    Is the on-site restaurant / canteen or local eateries likely to give everyone food poisoning (this could be a single point of failure)
    Local crime rate - number of times the facility has been broken in to - even the amount of graffiti on the walls could be a negative indicator

    --
    politicians are like babies' nappies: they should both be changed regularly and for the same reasons
    1. Re:Don't forget the non-technical bits by kroby · · Score: 1

      I assume you are looking to co-locate. If you plan on spending much time in the data center make sure that they have bathrooms available to their customers. Not all do.

  25. Everyone has the questsions... by Anonymous Coward · · Score: 0

    Everyone posts the "questions" to ask; no one posts the acceptable "answers" to those questions. Way to regurgitate uselessness, but I guess posting (even if it's useless) makes us all feel helpful/smart/educated/witty.

    - Ask them how many Amps their electrical cords are tested for.
    - Make sure you ask there EMI rating in Ohms / squared Newton times their rack cabling coefficient.

    Yes, I'm mocking you...

  26. You missed a few by syousef · · Score: 3, Insightful

    You forgot a few:

    - Enough qualified *on site* staff 24x7 to deal with all clients including yourself

    - 24x7 phone support, with people who understand English and have immediate access to the techies

    - Company financial records and history (You don't want someone almost broke or a new startup with no backing)

    - These days availability of virtualisation solution and supporting hardware (depending on your application, if virtualisation is an option)

    Oh and your emphasis on maintenance records may be a little misplaced. They can be faked. They also may not be available due to security concerns (of their other clients). *IF* you can get hold of them they should be complete. Hardware service level should be part of the agreement and service schedule should be part of that.

    --
    These posts express my own personal views, not those of my employer
    1. Re:You missed a few by Anonymous Coward · · Score: 0

      I concur with the financial records. We've been at a data center that was declared bankrupt; not cool if your server is impounded by the police, and needs to be freed by an administrative procedure which lasts more than 48 hours...

  27. an outside air duct by spywhere · · Score: 3, Informative

    When I worked at a corporate office in Maryland, they used the building's air conditioning to cool the server room.
    This worked well until the outside temperature got down to about 15 degrees Fahrenheit, but then it failed miserably: the outdoor condensers no longer functioned, the AC shut down, and the entire IT department went into a panic.
    The first time this happened, I (a lowly Help Desk tech) suggested to the CIO that he run a duct into the room from the outside: a simple fan would bring in enough sub-freezing air to cool the servers.
    The second time it happened, the look on his face told me he hadn't taken my suggestion seriously enough.
    The third time, he flipped a switch and the fan cooled his server room just fine.

    1. Re:an outside air duct by SuperQ · · Score: 1

      Yup, that was done at the University of Minnesota CS department a few years ago. They had to take A/C offline in the middle of the winter for a few days and the datacenter room was down 3 stories below ground. We ended up propping open 2 emergency exit stairs on 2 ends of the building, covering the two hallways with plastic tarps to prevent air flow. The the doors at either end of the datacenter were propped open and a gigantic 6' fan was turned on. It pulled a ton of cold air down one stairwell and pushed it up through the other.

      Worked great except for the grad students who had to sit next to the datacenter doors where it was now freezing cold 24/7 for security.

    2. Re:an outside air duct by Anonymous Coward · · Score: 0

      When I worked at a corporate office in Maryland, they used the building's air conditioning to cool the server room.

      This worked well until the outside temperature got down to about 15 degrees Fahrenheit, but then it failed miserably: the outdoor condensers no longer functioned, the AC shut down, and the entire IT department went into a panic.

      The first time this happened, I (a lowly Help Desk tech) suggested to the CIO that he run a duct into the room from the outside: a simple fan would bring in enough sub-freezing air to cool the servers.

      The second time it happened, the look on his face told me he hadn't taken my suggestion seriously enough.

      The third time, he flipped a switch and the fan cooled his server room just fine.

      the energy efficiency of the original solution is so bad that it should be CRIMINAL

      you not only provided a simpler solution that will work you saved him a BUNCH of money

      long live free cooling

    3. Re:an outside air duct by Anonymous Coward · · Score: 1, Informative

      I've also had to do this in a cooling emergency (which luckily happened in the middle of winter in Wisconsin) The important thing to remember is that you lose control of humidity, which is almost as important as temperature.

    4. Re:an outside air duct by Anonymous Coward · · Score: 0

      outside air through a duct & a fan? that can be dangerous. was it filtered for dust and humidity? if not, it might be a serious danger to your equipment.

  28. Personnel by girlintraining · · Score: 2, Funny

    More important than the technology is the policies and training of the personnel running the operation. It will fail, eventually: It always does, no matter how well its designed or what with promises of infinite uptime. So walk into the data center and count the number of people wearing hiking boots, divide by the number of racks, and there you go. The most grizzly looking guy wearing hiking boots usually knows everything. He also usually has a lighter and a screwdriver if you ask.

    I don't know why this is...

    --
    #fuckbeta #iamslashdot #dicemustdie
    1. Re:Personnel by Anonymous Coward · · Score: 0

      If he knows everything then it can be safely assumed the lighter is for those times when someone shows up with a PRD but not a lighter. He's smart, and he knows how to get in on a free doob.

      Sorry, no idea what he'd be using the screwdriver for ;-)

    2. Re:Personnel by Anonymous Coward · · Score: 0

      lighter for joint to deal with management, orange juice to protect against colds, alcohol to protect against freezing computer room.

    3. Re:Personnel by Anonymous Coward · · Score: 0

      HAHAHA This is so right, i've toured dozens of datacenters and this is what I always find:
      - A fat, hairy, untanned guy with a beard whos like the server crypt keeper
      - An indian guy named sameer or some shit like that, hes always got a phd from some ivy league school
      - A pale white guy with long hair in a pony tail who seems to know every IP & port off the top of his head

  29. a few things to think about by Anonymous Coward · · Score: 0

    Security:
    manned or unmanned 24x7
    Closed Circuit cameras
    How people are granted access to the facility
    Do you get a cage, a rack, or space in a rack?
    If a rack or space in a rack, how is your equipment secured?
    How do you grant access to vendors if they come to swap a hard drive or motherboard for you?

    Facility:
    How close are they to their maximum power consumption?
    How do they propose to scale once they reach maximum?
    KW per Rack limits
    How many power grids is the facility tied to?
    Localized fire suppressant or general "drench them all with water"?
    What kind of backup power generators and do they have access to fuel?
    Do they have a priority relationship with fuel suppliers in the event of an emergency?
    Do they have contiguous space available? If so, how much?
    What is their plan for growth if they fill all available floor space

    Network:
    How many carriers are present in the facility?
    Can you bring your own in, or do you have to share?
    what is the cost for cross-connects?

    Acct Mgmt & Processes:
    Billing process
    Do you have a dedicated account team?
    What is the process for dealing with SLA violations?

    Hands-on:
    How many hours and what skillsets come with the contract?
    What is the hourly rate to do simple tasks like swap tapes?

  30. do they let me tug my nuts? by Anonymous Coward · · Score: 0

    mmmmmmmmm raw

  31. Freight Elevator capacity... by HockeyPuck · · Score: 2, Interesting

    I used to have a large cage in an Exodus colocation facility. Turns out that if we wanted to put in an EMC Symm5 (these are three tiles wide), we would have to rent a fork lift and put it through an open rollup door on the second floor. Their "freight elevator" was barely big enough for two people and a dolly.

    One of my other cages was housed in a Global Crossing facility; when they started to run out of out cooling, they would hook up huge external A/C units in the parking lot and run 2ft diameter ducting to a hole in the wall. If you happened to walk near one of these openings you'd be greeted by freezing 50mph winds.

    Anybody find it odd that Exodus bought Global Crossing, who then went out of business?

    1. Re:Freight Elevator capacity... by georgewilliamherbert · · Score: 1

      I used to have a large cage in an Exodus colocation facility. Turns out that if we wanted to put in an EMC Symm5 (these are three tiles wide), we would have to rent a fork lift and put it through an open rollup door on the second floor. Their "freight elevator" was barely big enough for two people and a dolly.

      I bet I know what facility you're talking about.

      I put the first Sun E10Ks in one of those, when they were pretty new. It took a while to communicate "No, we're serious, it's that many inches wide, that many inches deep, and weighs 1,600 pounds. Show me your engineering drawings on the raised floor, and I need to measure the elevator again."

  32. I'm going to turn this around. by NoNsense · · Score: 5, Interesting

    I am the Director of Operations for our DC. When we give tours, I explain the following (pseudo order of the tour):

    - Begin with the history of the building, when it was built (1995), why it was build (result of Andrew in 1992), and how it is constructed (twin T, poured tilt wall).

    Infastructure:
    - Take you through the gen room, show you it is internal to the building, show you the roofing structure from the inside, explain the N+1 redundancy, the hours on the gens, when they are ready for maintenance, how they are maintained, by whom (the vendor), how the diesel is stored, supplied, duration of fuel at max and current loads. Explain conduct before a hurricane or lockdown, how we go off grid 24hours ahead of a storm, mention our various contracts for after storm refill and our straining / refill schedule.
    - Take you to the switch gear room, explain the dual feeds from the power company, how the switch gear works, show you the three main bus breakers, show you the numerous other breakers for various sub panels, etc. Explain and show you the spare breakers we have in case replacement is needed.
    - Take you to the cooling tower area, explain the piping, the amount of water flowing, the number of pumps, how many are needed, the switching schedule, explain the N+1 capacity and overall capability of the towers, explain maintenance, show you the replacement pumps in stock, explain the concept of condensed water cooling if needed.
    - Take you through the UPS and battery rooms, explain the needed KW capacity, what the UPSs back up and what they do not. Show the various distribution breakers out to floor, their capacity, the static switches, bypass, explain the battery capacity, type of cells, number of cells, number of strings, last time the jars were replaced and how they are maintained. Explain max capacity of the load vs time. Answer questions relevant to switching from utility->UPS->generator and back.

    Raised floor:
    - Take walk on raised floor, explain connectivity, vendors, path diversity we have, how the circuits are protected. Show them network gear, dual everything, how we protect from a LAN or WAN outage, and specific network devices we have for DDoS, Load Balancing, Distribution, Aggregation. Explain how telco and others deliver DS0 to OC-12 capacity, offer information on cross connections regarding copper, fiber, coax. Explain our offerings (dedicated servers up to 5K sq ft cages) and ask what they are interested in.
    - Explain below the floor, size of raise, that power and network is delivered under, what are on level one trays, level two trays, and the piping for cooling. Show the PDU units and how they related to the breakers in the previous rooms. Show them the cooling panel and leads out to CRAC units, explain the cooling capacity, plans for future cooling, explain hot/cold aisle fundamentals, and temperature goals. At this point, there are usually more questions about vented tiles, power types available and overall floor density in watts/sq ft.
    - Explain the fire detection / mitigation system, monitoring of PDU's, CRAC units, and FM200. Explain the maintenance of the fire system, show them the fire marshal inspection logs and the panels that alert the police and fire departments (both on floor and in our security office in front).
    - While finishing the walk on the floor, show cameras, explain process to bring in and remove equipment, tell them the retention on the video, explain the rounds the guards make, the access list updates and changes.

    NOC:
    - At this point we're back to the front of the building, go into the NOC, explain what we are monitoring (connectivity, weather, scheduled jobs, etc). Introduce NOC and security staff, explain they will always get a person if they call, submit a test ticket from a e-mail on my phone, they will see the alerts light up and the pager for the NOC will signal. The final steps are to introduce them to security and then I'll lead the customer(s) to the conference room so they can continue the conversation

    --
    So there.
    1. Re:I'm going to turn this around. by kilodelta · · Score: 1

      Nice job! I had the responsibility for redundancy planning too. Went with an APC Symmetra in room, and a 125kW natural gas fired generator on the back end with auto-transfer, regular generator exercising, and regular maintenance and monitoring as part of the whole APC package. Did a 10 year contract on that one. So they notice a battery gone bad in the Symmetra before we did. Also had a 480VAC dedicated service to the NOC.

      Dual redundant AC systems. One fails the other one runs.

      Raised floor (8") - this saved our bacon when we had to cable out 8 new servers. Luckily I know my way around the business end of a punch down tool.

      Separately keyed room locks. Separate alarm system on NOC. So both the I.T. space and the NOC had their own alarm systems.

    2. Re:I'm going to turn this around. by NoNsense · · Score: 1

      Our building is approx 66K sq ft. about 22K sq ft. is raised floor. We have three 1.25MW gens and three 500 KW ups units. The batteries are wet cell C&D with a 20 year life. We replaced all 135 jars last year. Three strings of 45 batteries each currently. We're moving to four 500 units and each will have 60 jars each. Everything is 480V, the next gen power room will be 13K gear input and stepped to 480 at the distribution. Right now, I'd consider us tier 2/3 but we have designs to go to tier 4. The original spec as 65w/sq ft and we've upgraded it to 125w sq ft. The max we can probably eek out is 200w sq ft. The second phase will be a 15K monolithic space with chilled AC units and slightly different power distribution. The other space will be rated at 250w sq ft up to 500w sq ft.

      I saw some of the other responses, in terms of certifications, years in business, redundancy in terms of data protection, etc. I think all of these questions are very specific. IF you want to eval a Datacenter in terms of management, track record, etc that's fine. But I think you need to generate a scope of what are the most important business needs you have. The outline I gave is generally the minimum I would go over, but the sales associate and customer service people go over many, many other things. Security is a huge concern, I'm happy to report that at least in our facility, we have a great security staff and NOC staff that give us 24x7 eyes. I think looking at the years in business is an OK measure, but more importantly is the track record of the people that run the show. Combined, I would say we have over 50 years between 3 people at the top, that doesn't count all of our techs and customer service. We have a solid track record, if it we're for Level3 and their inconsistency two years ago. Thank God we have NTT and Qwest as well.

      --
      So there.
    3. Re:I'm going to turn this around. by SHaFT7 · · Score: 1

      It is nice to read a comment by someone who definitely seems to know what they are talking about. Well written!

    4. Re:I'm going to turn this around. by NoNsense · · Score: 1

      Thanks.

      I've been in the biz for some time. Seen quite a few failures of other companies and it made an impression. I see there seem to be a bunch of Holy Wars going on in the thread "why do you need a DC, etc". I guess my response is, if you determine you need one, then the outline above is what I like to educate people on. Some people don't get it/can't afford it. Some boards of large corporations demand it, and sometimes, its a really difficult job when it comes to trying to meet every customers expectations. It's also very rewarding and challenging and I would do anything else with my time.

      --
      So there.
  33. a horror story by SendBot · · Score: 1

    I had a machine colocated in a very nice, secured facility right in the middle of a major city where all the telco wiring runs. It was awful for these reasons:
    - they advertised 24 access to your equipment on the web site, then the smarmy salesperson explained how that's actually not going to happen. That should have been it right there, but I was dumb.
    - later, they had a brief power outage due to a contractor f-ing up one day, and I was never notified. This in turn disabled my traffic shaping configs, which I intentionally do not have running upon startup. I didn't know anything was amiss until I got a huge bill for bandwidth overages. I had to fight heavily with them to overturn the charges because I had a contract with them that said 100% uptime in regards to power. They disabled my controls by not upholding their deal, and were trying to pin the results on me. Afterward I put in a simple script to email me when the maching mysteriouly rebooted. The whole time they were acting like jackasses about it, then acting like they were doing me a huge favor when they gave in.
    - Then, the worst offense. I took off for a three day weekend right when they cold-cocked my machine during a maintenance operation. I didn't notice until the following sunday that things had been down all weekend. I had missed my opportunity to visit my machine since it was just after 6:00pm, and I'd have to wait until 9:00 the following morning to see what the hell was wrong. Or they offered to let me PAY THEM to look at it for me sooner. I declined.

    When I got there, they had unplugged my machine, moved it to a new location, failed to power it back up, and had the network cable in the wrong port. All of those things were in total violation of my contract. When giving me excuses, they were saying that because the network lights were on they thought the machine was powered on.

    Then it was like all hell to get them to come through on their contractual provisions when they don't provide the guaranteed uptime and exhibited severe negligence.

    I eventually got paid back what I had paid for the service that month, but not any of the reimbursement specified in the contract for exceeding downtime. And it took two months for them to return my money.

    Anyone have worse stories?

    1. Re:a horror story by Anonymous Coward · · Score: 0

      You had critical settings that weren't implemented on startup
      Sorry but you deserve to be screwed!

      Do you trust cars to stop at a crosswalk just because you have right of way?
      There are a million reasons for a server to restart, everything that happens after that is your fault.

    2. Re:a horror story by SendBot · · Score: 1

      There are a million reasons for a server to restart, everything that happens after that is your fault.

      I had a contract in excruciatingly clear language that held them liable for the consequences of their failing to uphold their obligations. I also have very, very good reasons for not running that script upon startup, accidental or otherwise. Furthermore, that machine is used specifically for its redundancy and resilience in not mysteriously rebooting. I'm not running a google where machine failure is part of the formula, nor am I running crappy beige boxes on windows.

  34. And location factors too by Anonymous Coward · · Score: 0

    Is it near a hazardous materials facility? Under an airport approach or departure path? A rail line? A major highway possibly having traffic hauling hazardous goods? Gas, oil or chemical pipelines? Flood plane? etc.

  35. I'd be interested in... by v1 · · Score: 1

    - data redundancy, offsite specifically
    - ability to cut over? ie what happens if there's an earthquake, are your services to the world down until everything is replaced and backups are restored?
    - what do you have on hand for hot spares in the event of equipment failure?
    - when you are in failover mode for whatever reason, how does it impact your performance? ie does webmail just crawl until the mirror finishes rebuilding?
    - how are your external resources? got a plan to truck in gas for the genny if a tornado levels the local substation? got a hotline or multiple points of expert contact available 24/7 for every critical piece of hardware that you can't fix yourself regardless of how it breaks? same for software.
    - do you have a forensics plan in place? ie if you get hacked, (and don't answer that with "that can't happen") do you have any idea what you will do and in what order, to preserve forensic information, stop additional damage, and orderly cleanup? What are your legal obligations for notification, who is your contact with the press? (and there better only be ONE) Do you have a specific partner waiting in the wing all picked out if needed? after the fact is not the time to be choosing one.
    - if you have a failure that affects multiple services or clients, what is your priority order? who gets their service back first?
    - do you have a set "fire schedule" that people know specific additional hours they will be required to work in the event of an emergency situation? Are you going to run short on manpower in a specific area because you're already overextended by day to day operations in some aspect?
    - are there any people that are single points of failure? What if Bob gets hit by a bus? what if Dave is the only one that knows the firewall and gets hospitalized when it explodes while he's working on it? crosstrain crosstrain crosstrain.
    - not sure if it was covered above but documentation, documentation, and more documentation. How consistent is it? Does every network map look like it was written with a different drafting app by a different person? Is all of your documentation collected together and well organized? multiple copies in various places? are some things much better documented than others?
    - server rebuild lists. do you have a step by step set of instructions for EVERY critical box that will take it from a freshly formatted HD to back in production, that any of a dozen of your monkeys can follow, with no "well wasn't that obvious?" missing steps? And how often do you test these? Walk in one morning and drop a new box on a desk and say "WEB15 just got STOLEN. Rebuild it. Fast. Starting NOW." and hit your stopwatch and see what you get. You do this from time to time, right?
    - do you have a structured command that avoids differing opinions in a crisis slowing things down? when it comes right down to it there needs to be one clear person or command structure that has final say in a crisis.

    I'm sure I'm missing some things but that's a good start for ya.

    --
    I work for the Department of Redundancy Department.
    1. Re:I'd be interested in... by myowntrueself · · Score: 1

      If you weren't almost certainly in the USA, I'd say "Can I come work for you?"

      --
      In the free world the media isn't government run; the government is media run.
  36. PUE - Power Usage Effectiveness by SuperQ · · Score: 1

    Many good comments, but nobody is asking what PUE a datacenter gets. Bad PUE turns into lower rack deliverable power and more expensive power when you do get it. I would have a hard time picking a datacenter that didn't have tight closed loop hot isle cooling.

  37. References, references, references by trippd6 · · Score: 1

    The quality of a datacenter has less to do with the equipment (although thats important), and more to do with who designed and is running the equipment.

    Most of the datacenter outages I have been a part of in one way or another (Customer, or Provider) have been caused by:

    Poor planning
    Human Error
    Poor design

    As a normal customer, there is no way to know if any of these problems exist. The solution? Ask for references that utilize that datacenter. Make sure they don't give you a customer that utilizes another data center from the same provider. Data center design varies greatly, even across the same provider. Ask that reference how long they have been there, how many problems they have had, and the companies response to those issues. Look for a customer with a long history in that data center (3+ years, 5 would be better).

    Don't rule out a data center because they had an outage. Outages will happen, no matter how redundant their systems are. Their response to it is very important. If you find out about a previous outage, ask to see the root cause analysis they provided their customers. If they can't or won't produce it, even under NDA, then walk away.

  38. Only way to be sure.... by unitron · · Score: 1

    Nuke it from orbit and then see how soon their backup site with your backup data has you back online.

    --

    I see even classic Slashdot is now pretty much unusable on dial up anymore.

  39. Re:Get this out of the way by Anonymous Coward · · Score: 0

    I prefer calling them Congress-Hertz

  40. thoughts from long ago by 68882 · · Score: 1

    Some thoughts, having been a shift supervision in a DC long ago...

    (a) don't have the emergency poweroff & off halon dump switch uncovered right by the DC room's light switch. Someday someone will *hit* it...

    (b) where does the water go when the 5 inch chill water main fracture?

    (c) who is the cleaning staff and do they know not to turn the "computers" off when they dust? Also why are they dusting unsupervised
    in the data center anyway?

    (d) Oh sure you have multiple telco feeds that service the building from that *sole* pole over there, which that dump truck just took out.

    (e) And your procedure for the phone in fire alarm is what, wait for the fire dept to axe the entry door? This was my favourite. What's that chopping noise?

    (f) Lastly, if the city bomb squad runs a trial, let the staff know ahead of time because that ensures they might be able to focus after the squad leaves...

  41. my list: by Anonymous Coward · · Score: 0

    Some additional things I looked for recently when evaluating a colo were:

    - Ease of access to the building
    - logistics including:
      -- how is the parking situation? 24/7 parking?
      -- if i need to drop something off from my car, is it close to a loading dock?
      -- how are deliveries handled, where are they stored until i come in to use new said equipment
    - how many years remaining on the lease (assuming the building is leased)
    - when was the last time they checked their backup systems (power, cooling)
    - do they have multiple paths of entry for fiber into the building? (or will 1 backhaul bring down the interwebs?)
    - do they have crash carts? (monitor / key boards)
    - do they have spare cables or power tools I can borrow?
    - how sturdy are the racks bolted into the ground? (push them, see if they are telling the truth :) )
    - power density per rack that is available
    - cross connects to various providers
    - do i "like" the place ... this is subjective but important. talk to the NOC guys, are they nice, or do they hate their jobs, how long have they been working there, etc etc.

    just a few things I evaluated when choosing a colo.

  42. an important suggestion by Eil · · Score: 1

    Unless yours servers absolutely must be local, one of the most important factors should be local climate and environmental risk. I've worked in a couple datacenters in Michigan and it's really ideal:

    * No state-wide forest fires
    * No flooding if you're above the flood plain
    * No hurricanes
    * Very few tornadoes

    On top of that, if the AC units should spontaneously fail all at once, 99% of the time you can just open up all the doors and run a couple of large fans to keep things cool enough to run.

  43. One Server at a time by Anonymous Coward · · Score: 0

    All line up and evacuate slowly.

  44. Lol by Anonymous Coward · · Score: 0

    How dont you evaluate a data center

  45. Re:Get this out of the way by Hognoxious · · Score: 1

    You're focusing too much on the data aspects. How would you measure how central it is?

    --
    Confucius say, "Find worm in apple - bad. Find half a worm - worse."
  46. raised floor by Anonymous Coward · · Score: 0

    Newer datacenters don't have raised floors because it is more energy efficient to have concrete floors.

    Hogwash.

    Why? You can't put as much weight on a raise floor as you can on plain concrete.

    Further, if you're doing cold- or hot-aisle containment, you can do it without the need to do a raised floor. There are plenty of in-row cooling options so that you can put the cooling in the places you need it (either beside the racks, or on top):

                    http://www.42u.com/cooling/in-row-cooling/in-row-cooling.htm

    There's even in-rack cooling:

                    http://www.42u.com/cooling/in-rack-cooling/in-rack-cooling.htm

    Or, pipe the water (or refrigerant) straight to the aisle and put it in door-based cooling system which is rated to 35 kW per rack:

                    http://www.sun.com/servers/cooling/

    The above systems also don't have fans, so you don't have to worry about maintenance on it--just the overall circulation system.

    Not quite sure what the GP mean by the efficiency of concrete floors, but there are certainly better systems than general circulation air via raised floors.

  47. Evaluation of Data Centers is nothing new... by Anonymous Coward · · Score: 0

    Why re-invent the wheel?

    Yes it's People - Process - Technology... there are audit / checklist standards (multiple) for each area.

    Search for SysAdmin SA-BOK-0500.pdf and you'll find a good overall checklist. SANS Institute also has a physical security checklist as well.

    Beyond that you have the super serious audit things like SAS-70 or ISO-17799 / ISO-27000 or ISO-20000 or COBIT or ITIL... things that actually have outside audit standards / agencies.

    If you want the informal check lists for your own review, SA-BOK-0500 or SANS Institute is good. If you really want to do a proper / formal thing, inquire as to their SAS-70 and ISO-20000 (ITIL) compliance. Ask to see copies of latest audit under NDA.

  48. Some more points to consider... by GetSteved · · Score: 1

    Full Disclosure: I work for a (Great!) data center provider (ViaWest ).

    Infrastructure:
    - What is the UPS run-time?
    - What is the generator startup time?
    - What is the genset capacity in relation to UPS demand? (i.e. is the UPS demand larger than the genset capacity - you'd be supprised!)
    - Does the provider have multiple refueling contracts?
    - Are the refueling contracts high priority?
    - Can the provider detail out green initiatives to improve PUE?
    - Does the provider have sufficient capital resources to expand the data center?
    - How much investment has the company made - this year - into the data center?
    - Is the data center in a flood plain? Check http://msc.fema.gov/

    Compliance:
    - Is the data center SAS-70 type II audited? Type II means they're serious about it.
    - Are the results of the audit available for review?
    - Are a list of control objectives available?
    - How does the provider assist with customer audits? (i.e. PCI auditor requests for info)
    - Can the provider demonstrate servicing other companies where compliance is a requirement?
    - Will there be additional charges for audit related work or requests?

    Network Remote Hands
    - Does the provider offer managed hosting / hybrid hosting options
    - What is the expertise level of the NOC staff?
    - How are remote hands charged?
    - What is the response time for a remote hands event?
    - What monitoring options are available?

    Corporate
    - Does the company have a business continuity plan documented?
    - Are the company financials available for review?

  49. Carrier Neutral or Inclusive by dracocat · · Score: 1

    Depending on your uptime needs and size, also consider weather you want your Internet Access included as part of your Data Center or whether you want a carrier neutral facility. Many places just lump data access in with the colocation space and you get an ip.

    Other places, sometimes called a "hotel operator" simply rent you space and power, after which you can connect to one of usually a couple hundred ISPs that are cross-connecting in their meet-me room.

    Also, don't know if you are starting or moving. If you are just starting, be sure you look very closely into simply renting some cloud space. I admit I have been skeptical of it for a long time, but I am now a convert. Sure it is more expensive than buying your own servers and hosting them, but redundancy and capacity planning are almost eliminated.

  50. Well, how can I tell you ... by da5idnetlimit.com · · Score: 1

    Just to get it out of the way, yes, IAADRS (I am a Disaster Recovery Specialist - the "speaks bit and byte" and "cosfi" datacenter visiting type...)

    Concrete ? Well, yes. Under the raised floor. What did you want ? Marble ?

    No cooling from the raised floor ? why not ?
    Overhead network cables and a "newish" cooling solution like here :

    http://www.datacenterknowledge.com/

    front page. Think...giant heatsink...(first overlord joke gets the boot 8p)

    As for concrete, let me introduce you to this wonderful answer : it depends. Mostly on the concrete.

    "Ramazan Demirboa

    Civil Engineering Department, Engineering Faculty, Atatürk University, 25240 Erzurum, Turkey

    Abstract

    In this study, the effect of silica fume (SF), class C fly ash (FA), blast furnace slag (BFS), SF+FA, SF+BFS, and FA+BFS on the thermal conductivity (TC) and compressive strength of concrete were investigated. Density decreased with the replacement of mineral admixtures at all levels of replacements. The maximum TC of 1.233 W/mK was observed with the samples containing plain cement. It decreased with the increase of SF, FA, BFS, SF+FA, SF+BFS, and FA+BFS. The maximum reduction was, 23%, observed at 30% FA. Compressive strength decreased with 3-day curing period for all mineral admixtures and at all levels of replacements. However, with increasing of curing period reductions decreased and for 7.5% SF, 15% SF, 15% BFS, 7.5% SF+7.5% FA, 7.5% SF+7.5% BFS replacement levels compressive strength increased at 28 days, 7- and 28-days, 120 days, 28- and 120 days, 28 days curing periods, respectively. Maximum compressive strength was observed at 15% BFS replacement at curing period of 120 days."

    http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V23-4KPFKFY-1&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&_docanchor=&view=c&_searchStrId=1085368183&_rerunOrigin=google&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=b8e7b8ce7b5e23b07db5805bf9ad9740

    --
    It takes 40+ muscles to frown, but only four to extend your arm and bitchslap the motherfucker
    1. Re:Well, how can I tell you ... by Cramer · · Score: 1

      No cooling from the raised floor ? why not ?

      Not "none"; just not enough to matter. The TC of concrete is no match for the rate of heat generation in even a small data center. Take my little ~400sq.ft. data center... With the current hardware, the temp. rises 1F per minute that the compressor is off. That's with 2900ft^3/min of air flowing across the concrete floor (beneath the DC flooring.) Concrete simply cannot move that much heat fast enough.

      From a civil engineering point of view, it's an important consideration in any building design. A house or office complex will definitely see a difference due to the concrete. But their heat loads are significantly less. Just look at my townhouse sitting on a ~900sq.ft. slab (the entire community is built on slabs) -- approx. 2ft thick. In the winter, it's next to impossible to heat the place -- esp. with the hippie inspired crap hot water heating system [Apollo HydroHeat(tm)]. In the summer, it's hard to cool it. Yet in my former apartment, with wood flooring, it was trivial to heat and cool -- in fact, a half dozen computers kept the place warm as long as it was above freezing outside.

  51. Don't forget the contract details by mattmarlowe · · Score: 1

    * If you move your infrastructure in with them and later lose confidence in their abilities for whatever reason, how quickly and easily can you terminate and move out w/o all the hassle of pre-paying to end of term? This should be your biggest requirement. No matter how much do diligence you do on technical matters prior to moving in, things change...the datacenter provider could have financial problems, sudden facility problems, lose key critical staff, etc. Your contract needs to allow you the option to terminate when you reasonably lose confidence in their abilities w/o a billion hoops or outlandish costs.

    * Power pricing and limitations on how much they are allowed to pass on additional costs from utilities during term of contract.

  52. Why do you need a data center? by cryfreedomlove · · Score: 1

    Seriously. Use AWS for your custom apps. Outsource your email and other G&A aspects of running your company. Data centers are for dinosaurs. All of the cool kids are in the cloud.

  53. My only qualification for a datacenter by Anonymous Coward · · Score: 0

    My only qualification for a datacenter: Will it blend?

  54. Vending Machines!!! by chrisj_0 · · Score: 1

    See if you can eat for 2 days out of the vending machines!

  55. Simple answer by Anonymous Coward · · Score: 0

    A successful data center has no single points of failure. Make sure everything is triple redundant with a by without dropping the load.

  56. Re:Get this out of the way by Anonymous Coward · · Score: 0

    Libraries of Congress per second.

    Maybe not this figure, but it is definitely good to get statistics on load, figure out what all the systems are doing, etc.

    In the datacenter that I work in, there is a lot of deadweight, and even systems that are powered on, and doing nothing!
    Other systems are being hammered, and several of our tasks could *probably* have their load redistributed across some of the dead weight systems.

  57. ignore the magic of the City of Lights.. by yalap · · Score: 1

    Forget the presentations and the tours with all the impressive blinking lights tapering off to the horizon. It's all about what happens when things go wrong! We've had Alex answer the phone on a weekend and refuse to do anything with a windows server because he only likes Linux, wait for the Windows guys on Mondays. No help, no escalation. End of contract there. Failed mirrored disk? - don't be surprised if they replace the good one. Be aware that the fancy network status websites only include a sample of the actual errors, power outages and faults. Rebuild a simple server? three shifts to do it and the last guy left without leaving the root password. Ignore the data center review sites. They insist you provide a URL that they can track back to the hosting company's network. A brilliant way to ensure happy reviews only because no current customer is going to let the world know their servers are at a horrible hosting site. Make sure the escalation process works. Use fire drills on them to figure out if it really works. Another great question to ask - is their phone system VOIP? If they get a DDOS attack then you can't call them because it takes out their phone system too. The above mentioned analysis of power and cooling is wonderful but if the NOC manager decides not to pay overtime then maintenance gets done in business hours. These are not bottom-feeder companies, these are mid-range or higher price ranges. Either you have to deal with these problems and set the right expectation in their minds, or be prepared to have plenty of backups and be able to move to another company when required.

    1. Re:ignore the magic of the City of Lights.. by Anonymous Coward · · Score: 0

      1. If there isn't technical staff available 24/7 then you're an idiot for even considering that DC.
      1b. If a technician refuses to work on a system because of the is Windows instead of his favorite Linux distro, drop that DC.

      2. POTS -> Asterix -> VoIP == invulnerable to DDoS (unless you consider 50 customers calling at once to report poor network quality to be a DDoS), don't make such ridiculous statements.
      2b. If you see high packet-loss to your server, wait 10min before calling in, or you're just bothering support techs with useless questions/concerns while they're under a DDoS attack, if they cant fix the problem in 5-10min, consider a new DC.

  58. Less technical, than a feel for the place by funkboy · · Score: 1

    All the technical points folks are making here are very important.

    But the most important thing is the people managing the datacenter.

    At least in Paris, about 4/5 of the catastrophic failures experienced in the last several years have been due to:

    - the management being a bunch of slimy cheapos and not doing maintenance on time or cutting corners when they do get around to doing maintenance

    - some cro-magnon "technician" from a maintenance contracting company doing something stupid because he was completely unsupervised

    In all cases, these datacenters had everything that folks above have described: dual theoretically diverse utility power feeds, dual generators with big fuel tanks, big battery systems, dual theoretically diverse chiller circuits, etc etc etc.

    The only thing you can do to protect yourself against this sort of thing is to treat the datacenter selection process, especially the salesbeasts, as a job interview. If you say something like "I don't care how big your generators are; show me proof that they've had an oil change sometime in the last two years, that you test them regulary, and that your emergency fuel delivery contract is paid up" and they bullshit you, it's time to look elsewhere.

    The single most important thing for me is to find out what procedures they follow when a 3rd party contractor is on-site doing maintenance on their critical equipment, especially the power transfer systems. Power control master switches seem to have some sort of special attraction for morons. Outages experienced recently (all the fault of the unsupervised 3rd party maintenance technician):

    - removing both utility feeds from the master control switch along with both of its own internal battery backups at the same time, so that it defaults into the fail-safe mode of "off"

    - needing to transfer a PDU feed from one source to another and being alone, so he shuts off the primary feed before he walks over to the backup to enable it

    - doing some kind of "test" of the redundancy settings and screwing things up enough that the datacenter power is running off house batteries, the generators do not kick on as they detect that the utility power is working, and the batteries are disconnected from utility power. 15 minutes later, all the batteries are flat and the datafloor power is dead. The house lights still work as they run off "dirty" power direct from the utility, and the cleaning people are running the vacuum.

    - during a cooling system purge, leaving the drain valve for the cooling system open, with the fill valve for the reserve water tank shut, and the reservoir level alarm disabled, broken, or ignored. It took almost 24 hours to get the datafloor temp back to normal as the entire cooling system circuit was dry.

    - some construction jackass lucky to still be alive drilling directly into the master B-feed power riser cables and getting God knows how many amps directly into his concrete drill. Every single individual breaker in all the B-feed PDUs on every floor popped. The worst bit was that the jackasses that run the place didn't have a master record of the breakers in each PDU (the data was just kept in the individual client records), so they started digging through all the client records one by one (also note the lack of someone that knows about SELECT FROM) to figure out who to turn back on until about 10 people ran into the DC manager's office screaming at them to turn everything on and sort out which ones they should turn back off later.

    - a maintenance by a utility power technician causing the datacenter power system and the utility power system to have a somewhat different idea of what constitutes neutral voltage on a ground, again leading to the generator system thinking that the utility power was just fine but the battery system detecting a ground fault and refusing to us

  59. Reboots by Anonymous Coward · · Score: 0

    Good thing to check is what is the average time after submission until a reboot is completed. Along with this do they have a SLA for reboots and what the price is for the reboot.

    Also if you might need remote hands(if you are not within driving distance to the DC) how long do they take to attach the equipment and at what cost for what length.

  60. After u are done with all these rigorous questions by Anonymous Coward · · Score: 0

    After u are done with all these rigorous questions

    print them out and ball them up and throw them away.

    Because either you (and/or the money people) will decide based on completely unrelated criteria or the data centers won't give you half of the info you requested (maintainance records? 9/11, total power in vs current used? 9/11 and trade secret, etc) or if by some miracle they do give you the info then it means basically jack.

    Take the pompous asses at 365 Main for example, if you go by all this crap that the other people here are telling you then they would be near the top of the list - N+fucking 1, etc but guess what? They had a huge outage due to, as they tell the story "commands hiding like scared bunnies in the command queue" of their uber N+1 flywheel generator systems who came scurrying out when they went to fire up the generators. (Yeah, right - we bought a $125 million colo for $2.6 million, waaaahoooo!! and the vendors and contractors were left holding the bag for the difference - but we charge you like we spend $125 million - suckers)

    Meanwhile, you have a colo nearby (now web.com, previously Verio) that had not had an outage in 4 or 5 years but didn't have all the buzzwords.

  61. One important criteria that needs to be met by waferbuster · · Score: 1

    Based on a recent video on slashdot, the datacenter needs to be located other than in a flood plain.

    --
    I'm an individual! Just like everyone else!
  62. other stuff by mlg9000 · · Score: 1

    Besides the obvious backup power, cooling, environmental stuff mentioned above....

    1. Expandability.. how much rack/cage space is available nearby? Get a right of refusal on any empty space near your stuff if you can.
    2. Power max per rack, can you get enough for SAN's, blades etc
    3. Remote hands availability/skills/costs (really want to make a trip to the datacenter to replace dead hard drives? No. Do the employees know enough to do *limited* work for you)
    4. 24/7 access, near employees if a physical presence is required
    5. What sort of racks do they have? Can you buy your own?
    6. Storage space. Need 100 servers shipped but don't have racks yet? Need to keep spare hardware on hand? Do they have room keep that for you?
    7. How many/what carriers do they offer? How is access delivered? What does their network look like?
    8. Do they have spare tools/network cables/misc parts. Can they order stuff for you, or is there some place nearby you can pick things up if needed in a hurry?
    9. How many employees can you get access for?
    10. Do they have a crash cart? Comfortable place to work? Wifi or other forms of internet access available?

  63. Re:sell:shoes,handbags,T-shirt,Jeans,sunglass by Anonymous Coward · · Score: 0

    I'm looking for a datacenter. You got any of those for sale?

    Dick.

  64. watch out for power sharing agreements w/ utility by Anonymous Coward · · Score: 0

    if utility power goes out, batteries and Gen kick in.... but if the center is on gen power as part of peak load sharing with the utility... and the fuel runs out, can it auto kick over to the grid? Happened at an internal telco data net ops site... took two weeks to get everything right... :-)

  65. Step #1 = DEFINE YOUR NEEDS by lanner · · Score: 2, Insightful

    I'd guess 90% of projects fail at step #1: Define your needs. What's the objective here? Why are we doing this, and what are the benchmarks required for success. Does this sound familiar?

    First, define your needs, then evaluate possible solutions to what might meets your needs.

    If you don't know what you need, you don't know what the hell you are doing. Hire someone who does, like a consultant.

    1. Re:Step #1 = DEFINE YOUR NEEDS by Anonymous Coward · · Score: 0

      Spoken like a consultant. Of course, it is well known that consult is a contraction of Con and insult.

    2. Re:Step #1 = DEFINE YOUR NEEDS by Mr.+Freeman · · Score: 1

      On the other hand, it's also spoken like someone with some idea of what he's doing.

      If you need something that is outside your area of expertise, then you hire someone who knows what he's doing. For example, let's say I have an idea for a website and I know how to run all the servers and deal with the IT side of things, but can't actually design the website to save my life. I then hire someone who knows how to design and code a website.

      --
      -1 disagree is not a modifier for a reason. -1 troll, flaimbait, redundant, overrated are NOT acceptable substitutes.
  66. Data security paranoia by turing_m · · Score: 1

    If you have commercial information that you absolutely cannot allow to fall into the wrong hands (or accidentally deleted, corrupted, not backed up, whatever), is storing that data in a data center ever really acceptable? I would think not, but I'd like to hear someone else's opinion. Has anyone here done things DIY for this very reason?

    --
    If I have seen further it is by stealing the Intellectual Property of giants.
  67. How to choose a colocation facility by eprosenx · · Score: 2, Informative

    I wrote an extensive article on choosing a datacenter/colocation facility several months back. The full post can be found on my blog, but I will paste it below for your Slashdot reading convenience:

    http://www.bitplumber.net/2009/04/how-to-choose-a-colocation-facility/

    How to choose a colocation facility

    Choosing a colocation facility is one of the most important decisions an IT professional can make. It will have repercussions for years down the road, as there is generally a contract term associated, and it becomes difficult/costly to move. At the same time, unless you are a facilities professional, it is hard to tell the difference between the quality of one facility vs. that of another without knowing the right questions to ask. I have developed this list in the hopes that it will be a reference to folks evaluating datacenter options. This has been written using the assumption that you need a local datacenter rather than a DR facility (which can have very different needs), however, many of the same concepts will apply.

    Location
    When it comes right down to it, there are still certain things you have to do physically in person. You can’t run a network cable through SSH or RDP. Having a datacenter close by makes a huge difference, especially when you lose remote connectivity and must go push a button in an emergency (we all have done this once or twice). In general, the newer, more high-end, and redundant your equipment is, the less you should have to touch it in person. Things are getting much better with out of band remote access controllers, but sometimes being there is worth a lot. You can’t hear that fan making funny noises from your office.
    Does the facility have good access to transportation such as freeways and airports? Are their hotels nearby if you will have out-of-town contractors visiting? How close to logistics depots are you for your vendor-of-choices parts, i.e. Cisco, Dell, HP, etc
    Does the facility have adequate parking that is close to the building, does it cost money? Is it somewhere you want to leave your car in the middle of the night while you are inside working?
    Do you have line-of-sight to the datacenter? If you can manage to get a wireless link to your datacenter this can be an extremely cost-effective option for high speed connectivity. There is something to be said for controlling your own destiny when it comes to your connectivity rather than being at the mercy of a telecom provider. Will the facility allow you to put a wireless antenna on the roof and how much will they charge?

    Staffing
    Do they have on-site staff 24×7 to respond to emergency situations, to secure the facility, and to provide access when you forget/loose your badge (or have to stop by on your way home from the gym).
    If they do not have staff on site 24×7, what is their on-call policy? How long would it take them to respond to a power failure, a UPS exploding, a transformer catching fire in the parking lot, an Internet outage, an FM-200 fire suppression system going off, an HVAC system failing, or any other major malady (yes I have had all of these things happen to me in facilities I have worked in, and I am still waiting for the day a fire sprinkler goes off or there is a real fire in a datacenter).
    What level of professional services can they provide? Basic remote hands (please press the power button)? More advanced troubleshooting (help diagnose a failed network switch)? Or even managed services (i.e. they take care of backups).
    How competent are their NOC engineers, facilities folks, etc What quality of vendors do they use to do electrical work, HVAC maintenance, network cabling? This can be hard to tell, but there are lots of small clues you can pick up on.
    Does their staff speak English fluently and without heavy accent? It is extremely difficult to communicate on the phone with someone in a loud datacenter environment about complex technical issues when both of you are having a hard time understanding each other. This dramatically slo

  68. the quality of the toilet paper by brak · · Score: 1

    if they care enough for you to have an extraordinary experience when you have to use the restroom, that means they really care.

  69. Working Conditions by Deal-a-Neil · · Score: 2, Informative

    Is there a good desk working area? Is there a landline/PBX for you to make calls from? Is there decent mobile phone reception in the work area and by your cabinet? Can you eat food or bring drinks into the work area or around your cabinet? Is it in a shady neighborhood, where you might feel a little intimidated bringing in tens of thousands of dollars of emergency IT equipment @ 3 AM? In the event that your credentials aren't working (i.e. hand scanner, ID card swipe), can they let you in remotely, or is it manned 24/7? Is it carrier neutral and are there other backbone providers that you can connect with? Do they charge for running cables between cabinets, especially in cases where the cabinets are not adjacent? What is the max amperage that they'll provide per cabinet? Do the rack cabinet doors remove easily? Are there chairs available, and damn it, are they comfortable?

  70. Are you buying the DataCenter or Renting Rackspace by jgalietto · · Score: 2, Informative

    Your question is a little ambiguous. Are you looking to buy a data center of your own or are you renting rackspace?

    If you are buying the Data Center
    1.) Normal title , lien, Structural due diligence as for any RE purchase
    2.) Is it on a flood plain
    3.) Seismically active site?
    4.) Serviced by multiple communication providers from multiple CO's
    5.) Power available from two different substations.
    6.) Physical security / susceptibility to civil unrest
    7.) Physical access driveways, parking, loading docks, hallway widths elevators ramps
    8.) Floor / raised floor design loads. I have seen more than one raised floor rippled by rolling overweight gear on it.
    9.) On site power generation / fuel storage. Mech. condition, age, availability, reliability, repair-ability
    10.) Sufficient Chiller Capacity
    11.) Sufficient UPS / Power Conditioning
    12.) Sufficient space both for current needs and growth for planned lifetime
    13.) Sufficient office / command center space

    Those should be adequate to get you started.

    For rented rackspace
    I would say you at least need to glance at items 2 through 11 above. Beyond that
    1.) Per rack power limits
    2.) Physical security
    3.) If you are using "hands on" services it's skill set and response time.
    4.) Whatever value add services you will be using.

      Sorry it is late and a long day and this is all I can think of.

  71. It depends by ctrl-alt-canc · · Score: 1

    If you need a data center for your business, and you are into one of those Ponzi-like schemes, the first thing to look at is the speed of low-level disk formatting, just in case...

  72. White Mountain datacenter by G3ckoG33k · · Score: 1

    White Mountain datacenter in downtown Stockholm, Sweden. It is located in a bunker 30 meters under solid bedrock. It was a cold war bunker that was converted into this datacenter and is said to be able to withstand a Hydrogen bomb blast. http://www.youtube.com/watch?v=qwlATf9xse4

  73. The things they won't tell you by Anonymous Coward · · Score: 1, Informative

    All the normal stuff is important, but it is the things they won't tell you that is really important.

    I visited a data center that was really easy to commute to. See, it was above a subway train. Further, it was below a parking garage. Nice.

    I've seen data centers at the end of airplane runways. I've seen data centers in flood zones, hurricane areas, earthquake prone locations and with roof leaks. Nobody is going to point these things out to you.

    I really like how one DC manager showed off the huge springs that the turbine generators were sitting on to prevent vibration translating to other parts of the building, but they were behind on their battery maintenance schedule. Doing the simple things, well and consistently are key.

    In the end, you can't trust anyone and the safest answer is to spend your money in 2-3 locations. Don't make 1 a "primary." They all need to be "primary." Mix production, DR, Dev and Test apps across all locations, split them evenly, and keep them 500mi or more apart. Don't have a single supplier for the DCs either. Consistency is good, unless it isn't. Being different in each location and having different management is a hassle, but both probably teams won't make the same critical mistake that takes your systems down.

  74. Don't hire, Wesley Carver by G3ckoG33k · · Score: 1

    Don't hire, Wesley Carver! http://www.amazon.com/review/R3VO962Z99T6KZ

  75. Don't forget to read the small print by Anonymous Coward · · Score: 0

    What kind of contract must you sign? Is it in legalese or in understandable English? Will the data center only be liable for broken hardware in case of a failure or will they also compensate for your loss of revenue and blemished reputation?

  76. Things you DO NOT want to see in a data center by Anonymous Coward · · Score: 0

    1. water based fire suppression (sprinkler heads over server racks)
    2. security fences that only go up to a false/lowered ceiling
    3. metallic dust on surfaces (from cutting metal in DC)
    4. external/temporary/emergency cooling ducts on DC floor blowing cool air from AC unit parked in loading dock
    5. generator running for 2 weeks
    6. tropical feeling to air in DC
    7. 45 minute wait to get access to customer colo area due to no onsite staff
    8. power conduit under raised floor obstructing airflow
    9. fire suppression override always set to off
    10. mis-configured power ATS
    11. NOC staff that are active crackers / black hat
    12. hill billies doing rack installs / construction
    13. windows server datacenter edition

  77. don't forget by Anonymous Coward · · Score: 0
  78. What you want is two (or more) data centers. by cenc · · Score: 1

    I am sorry, but all these people going on and on and on about what you want in a data center are missing the unforeseeable. And the only way to do that is redundancy. What you want is two or more different data centers, in two or more distinct regions (ideally of the World).

    Ask yourself, what would you want running if the entire city or region was nuked, had an earthquake, was hit by a Tsunami, or an asteroid dropped on it? How about if an airplane flies in to it?
    What happens if the regional power grid or network is out for days, weeks, or months? Ask all the guys recently in Asia or the middle east about underwater data cables being cut, and what they wish they had. All the above have happened, and will likely happen again. Do you really want all your eggs in one basket?

    The only way to be sure is to diversify your data over as large a geographical area as possible.

    I will still take larger numbers of lower quality but diversified data centers in numbers, over a single high quality data center any day. Of course, having both helps.

    By the way I keep my data on four different machines, located in four different physical locations in two countries, on two continents. I figure any disaster that will take all four down, will be one sufficiently big that I just don't give a dam anymore.

    1. Re:What you want is two (or more) data centers. by Mr.+Freeman · · Score: 1

      "Ask yourself, what would you want running if the entire city or region was nuked?"

      If part of the US was nuked then your servers matter exactly nothing. I don't understand your comment about having redundancy in different areas of the world. Any event large enough to affect an entire country is large enough to affect the world to the point that your business matters nothing. Global free-for-all war? Yeah, no one's going to be visiting your website to check up on the latest furniture prices.

      --
      -1 disagree is not a modifier for a reason. -1 troll, flaimbait, redundant, overrated are NOT acceptable substitutes.
    2. Re:What you want is two (or more) data centers. by cenc · · Score: 1

      The World has survived nuclear attack. It has even survived world wars. A lot of data did not survive those.

  79. Green & be pragmatic by kelf · · Score: 1

    Data center draws a lot of power. One should consider whether there are Green technology implemented to reduce operating expense. Also spelling out stringent resiliency requirement is good, but one must be pragmatic when doing the specification. What are we trying to protect, what is the business impact, how much downtime can the business tolerate, and what level of risk can the business take. When carefully thought through, the added resiliency might not be necessary.

  80. Cooling failure modes by Nicolas+MONNET · · Score: 1

    This summer Equinix Paris had a major failure of their cooling system. Of course, they had a backup, but as was expected, the backup was identical to the primary system, and therefore failed identically. Temperature raised over 55C AFAICT. We didn't experience hardware failure since all our servers shut down automagically at 45C. We also had all our systems clustered over a gigabit MAN to another DC, so we suffered only a minor outage.

    Shit happens. You always have to keep that in mind. But two things could have made the whole situation better.

    - They didn't alert us of the failure. We could have avoided the outage completely had we been called when the cooling system failed, instead of finding out an hour later. We could have supervised the switch over, and there would have been no loss of service at all.

    - Datacenters should not get that hot, ever. They should be built so that they don't get much hotter than outside temp; it's not that hard, thanks to a high technology device called a fucking WINDOW. Too hot? AC down? You open the god damn windows! I believe the latests DCs are built that way, with passive cooling doing most of the job.

    You don't need high end AC with temperature controlled down to the degree. It's a complete waste of money. Instead you should have reliable AC, with graceful failure modes.

  81. Even the best fail by ebvigmo · · Score: 1

    Just wanted to comment that the data center I hosted at for 7 years (Chicago Equinix via Internap) had all the right stuff. But when the power to the facility went out when a fire broke out at local power station, one of the UPS generators failed when the wires shorted out. The other generator (it was an N+1 facility) couldn't handle the load and the whole facility was down. Good thing for us it was on a Friday night which was a slow time for our ECommerce site (ArtSelect.com, now shutdown by art.com who bought us).

    --
    CTO using Perl and MySQL (and proud of it).
  82. Re:Get this out of the way by rwa2 · · Score: 1

    If "pro" is the opposite of "con", what's the opposite of progress?

    Anyway, I lol'd at the Libraries of Congress per second thing... shame the downmodder didn't "get" it.

    I live in the DC area and drive by the National Archives every once in a while, and I still hate it when it's used as an analogy.

  83. Read the Google book on that topic by Anonymous Coward · · Score: 0

    Read Urs Hölzle and Luiz Barroso's (both Google) book on datacenters:
    http://books.google.com/books?id=Y3hhzdSSK58C&dq=h%C3%B6lzle+barroso+warehouse&printsec=frontcover&source=bl&ots=3GAME9c91o&sig=SFKN5MEnDyxy_Zk_bOctH776cjU&hl=de&ei=xaz5Su-zB8fm-Qbur_ngBw&sa=X&oi=book_result&ct=result&resnum=1&ved=0CAsQ6AEwAA#v=onepage&q=h%C3%B6lzle%20barroso%20warehouse&f=false

    Also available as PDF somewhere.

  84. what about the human aspects? by sku158 · · Score: 1

    i did not read all the comments but i had done cage wrestling with servers as a week long event as well as those mid-night cravings that i need to touch the servers just because they are not purring at the right frequency. in addition to all the power, connections, server survival conditions, server security, assisted monitoring (don't rely solely on anyone), etc etc. i also find the need for good nearby parking, large clean bathrooms, available stools or chair, tool set(sockets, big screw drivers, pilers), a place for a coffee break, scream at the VAR and making excuses to the spouse, ULTRA HELPFUL. for small setups and fuzzy architects, these are something as a tech look for in my experience.

  85. Comment removed by account_deleted · · Score: 1

    Comment removed based on user account deletion

  86. Read the fine print by efalk · · Score: 1

    While everybody else is suggesting technical things to look at, I'm going to suggest that you look at the legal things instead.

    Read the contract with a magnifying glass. In fact, have a professional with experience look at it for you. The contracts often have traps in them that it takes a trained professional to spot.

    For example: one managed hosting provider had it in their contract that they would replace any dead servers within 2 hours of determining what the problem is. Catch is, all they had to do was say "we haven't determined what the problem is yet". Kept our main server off line for nearly a month while the night crew and the day crew argued over what needed to be done.

    Contract said that if you so much as upgraded a disk drive, that re-upped the contract for eighteen months. Think about that for a bit. We couldn't afford to extend our contract for another 18 months and we couldn't do a thing about our overloaded server situation. You also had to give 60 days notice before the end of the contract or it was automatically renewed.

    Also: never go with just one provider. Diversify. If all your data is in one data center, they can hold it hostage. Happened to my dad in the mainframe days, happened to me last year. Some things never change.

    I could go on for hours, but I need to watch my blood pressure. Or my cholesterol. Something like that anyway.

  87. Easily by Anonymous Coward · · Score: 0

    In down-time per megaton-kilometer.

  88. How to find a new data center... by BSG1 · · Score: 1

    Check out http://www.usdatacenterlist.com/ - from there you can search and compare data centers on their technical and operational processes to ensure they are a good fit before you engage sales staff!!

  89. Another important topic by Anonymous Coward · · Score: 0

    There are many good points above about infrastructure and convenience, all should be considered when choosing a data center.

    However, there is another point which should be considered; how well is the company run from an operation/core values standpoint. You can have the coolest features in the industry, but if the culture of the company doesn't invoke 100% accountability throughout the entire organization it doesn't mean much. Make sure that whatever company you choose has the correct value from all aspects of the business, not just data center functionality.

    http://www.onlinetech.com/company/core_values/

    Geographic location is also a necessary component of your choice. The Midwest is typically the best choice for a number of reasons. Here are a few:

    http://www.onlinetech.com/company/company_overview/

    Jason Yaeger
    Online Tech, Operations Manager

  90. Useful link from an industry veteran by 1sockchuck · · Score: 1

    John Savageau, who has been an executive with a number of hosting and colo companies, recently blogged about Questions Data Center Operators Don't Want You to Ask, which some colo shoppers may find useful. It looks at issues for colo centers in mixed-use buildings and the merits of SAS70 certifications, which are often a key marketing point for facility operators.

  91. External hinge pins? Outsourcing oversight? by Anonymous Coward · · Score: 0

    My auditor is really nice, but some things in a data center really bug them. And, that makes my job difficult.

    Can you tell me if my auditor will like visiting you? My auditor doesn't like:

    - external door hinges they can pop the pins out and get directly into the "secure" server rooms from the parking lot. Especially the ones on the roof and fire exits.

    - outsourcing without asking if I'm OK with that, and ensuring we all know how we will manage any added risks.

    - security personnel vendors that can't provide the last date they verified all their temporary contractors and terminated employees had their access shut off.

    - same for data center personnel; and saying it is not the data center's responsibility to oversee the data center's vendors.

    - propped doors for smoke breaks with no alarms.

    - battery-powered wireless security features that don't tell you when the battery is dead.

    - preventive maintenance personnel with 24x7 access cards.

    - if I get all I need via email to verify, but that our auditor will have to provide 2 weeks notice of all questions, be accompanied by me, make no audit records (i.e., photocopy, pen
    and paper, laptop typing, etc.), bring no equipment, not write anything down, not ask any questions involving HR or outsourced personnel, etc.

    - the ability to plug in a USB drive or USB modem in a server without detection

    - not allowing me, my auditors, or an independent auditor data center information for "security reasons" to validate our GLBA, HIPAA, FISMA, etc. compliance

    - not having an online data center ticket request and clearing system that I can watch for progress and compliance status.

    I assume you know the rest like uptime, capacity, insurance, etc.

  92. Data Center Evaluation White Paper and Workbook by Anonymous Coward · · Score: 0

    I work for FORTRUST, a data center services provider in Denver, CO. FORTRUST published a comprehensive white paper and workbook as a data center site selection resource. The white paper, Evaluating Data Center High-Availability Service Delivery, examines several key criteria such as operational process controls, service assurance, maintenance and lifecycle strategies, critical infrastructure management, and capacity planning. Download the free white paper and workbook at http://tiny.cc/yqn4C.