Slashdot Mirror


Open Compute Developing Wider Rack Standard

1sockchuck writes "Are you ready for wider servers? The Open Compute Project today shared details on Open Rack, a new standard for hyperscale data centers, which will feature 21-inch server slots, rather than the traditional 19 inches. "We are ditching the 19-inch rack standard," said Facebook's Frank Frankovsky, who said the wider design offered better heat removal and a unified approach to power, including a 12 volt busbar. The Open Compute Project, developed by Facebook to advance open source hardware design, believes an open approach can avoid the mistakes of blade server chassis design."

44 of 237 comments (clear)

  1. Clouseau: The case is solv-ed by alphatel · · Score: 4, Informative

    So put the server power supply on the outside, basically.

    --
    When the foot seeks the place of the head, the line is crossed. Know your place. Keep your place. Be a shoe.
  2. metric? by SkunkPussy · · Score: 5, Insightful

    The first mistake is that they are still talking in inches instead of metric units.

    --
    SURELY NOT!!!!!
    1. Re:metric? by Anonymous Coward · · Score: 3, Insightful

      Arguing metric vs. imperial units is pretty much the epitome of bikeshedding.
      We can do arithmetic nowadays.

    2. Re:metric? by Christian+Smith · · Score: 4, Funny

      Arguing metric vs. imperial units is pretty much the epitome of bikeshedding.
      We can do arithmetic nowadays.

      We can do, but people (even rocket scientists!) still get it wrong: http://en.wikipedia.org/wiki/Mars_Climate_Orbiter

      Technical issues should use SI units.

    3. Re:metric? by Anonymous Coward · · Score: 5, Insightful

      http://en.wikipedia.org/wiki/Mars_Climate_Orbiter Technical issues should use SI units.

      The Mars Climate Orbiter was a case of someone not labeling their units. The unit system wasn't the problem.
      Secondly SI isn't always the best unit of measurement for performing calculations. In plasma physics we use eV in stead of joules for energy because it simplifies our work. In astro physics measuring distances in the SI unit of length, the meter, is impractical. If you are doing anything with relativistic electro magnetism, it's best to do your calculations in cgs because it eliminates useless constants.

    4. Re:metric? by MetalliQaZ · · Score: 3, Funny

      Actually, I think deliberations over the color of a bike shed is the epitome of bikeshedding.

      --
      "Here Lies Philip J. Fry, named for his uncle, to carry on his spirit"
    5. Re:metric? by richardkelleher · · Score: 3, Insightful

      The first mistake is that they are still talking in inches instead of metric units.

      This was my thought exactly. If we continue to build new standards around obsolete measurement systems we are just pandering to the Luddites. It is time to move America forward into the 19th century. If we can't engineer for the 21st century, we should turn the creation of standards over to people can.

    6. Re:metric? by ChatHuant · · Score: 2, Informative

      The Mars Climate Orbiter was a case of someone not labeling their units. The unit system wasn't the problem.

      But units wouldn't need to be labeled if everybody used the same system. The continued existence of the zombie Imperial system is the root cause of the problem.

      Secondly SI isn't always the best unit of measurement for performing calculations. In plasma physics we use eV in stead of joules for energy because it simplifies our work. In astro physics measuring distances in the SI unit of length, the meter, is impractical

      Which is why SI has a number of accepted units. You'll note that both the eV and the astronomical unit are there, but not the feet or yards used by Lockheed to send a rocket past Mars.

    7. Re:metric? by David+Chappell · · Score: 5, Insightful

      The first mistake is that they are still talking in inches instead of metric units.

      I would imagine that the reason is that 21 inch racks are already a standard. They are widely used in telephony. Introducing a third standard (say 55 cm racks) would likely complicate things for little gain. Better to just call them 5334 mm racks.

      This has already been done repeatedly in engineering. When I visit Europe (I am an American) I see things in Imperial units all around me, but most of the time nobody but me knows that they are. 6 mm steel cable? Why not 5 mm? Because 6mm is 1/4 inch. While is their 900 grams of buckwheat in this bag rather than 1000 grams? Because it is a two pound bag filled eight grams short. Railway gage in the former Soviet Union? 1524 mm. Yup, thats five feet. See that 1220x2440 mm sheet of plywood. Yup, 4 by 8 feet.

      This by the way is why it is so hard to convince Americans that they should switch the building and engineering trades to the metric system. All our building materials are sized on Imperial units. Converting these dimensions to oddball metric sizes would mean that builders would need to memorizes all sorts of weird dimensions and carry pocket calculators to figure out where the center of something is. There have been many serious attempts to do engineering projects in metric units. First they find out that they can't get materials in even metric dimensions. Then the vendors ask what they mean when they order 914.4 mm doors. Then the builders ask to have the drawings converted into 'the real measurements'. It begins to seem too much like masocism and on the next project they go back to units everybody understands.

      Many professions and markets have converted to metric in the US. Medicine mostly has. So has alcohol distribution. So have laboratories. So have our automobiles. But, I don't see any way for the building trades to convert.

      The metric system is clearly superior in scientific contexts. There the argument about ease of conversion is powerful. However this argument is meaningless in daily life. The only unit conversion the average person does is feet to inches. You don't need to know how many feet are in a mile in order to know whether you are exceeding the speed limit.

      I have found one use for metric units in daily life: increasing ones 'geek cred'. Since expressing dimensions in metric units is a form of elitist obfuscation, it is a great way to be annoying.

    8. Re:metric? by Anonymous Coward · · Score: 5, Insightful

      But units wouldn't need to be labeled if everybody used the same system. The continued existence of the zombie Imperial system is the root cause of the problem.

      Which is why SI has a number of accepted units. You'll note that both the eV and the astronomical unit are there, but not the feet or yards used by Lockheed to send a rocket past Mars.

      ALWAYS LABEL UNITS! This was the ONLY cause of the problem with the mars orbiter. If Lockheed used kilometers as their units instead and Nasa assumed meters the probe still would have had problems. If I tell you I have an energy of 5 does that mean 5 eV or J? You don't know if I don't label it.

    9. Re:metric? by spire3661 · · Score: 2

      Logically false. You are saying that the existence of a different measuring system is the cause of the human failure to differentiate. It was a human failure, what you are asking for is to dumb it down so humans cant fail in that way anymore. I assure you, humans will find some other way to foul it up, no matter how many rubber bumpers you put on things.

      --
      Good-bye
    10. Re:metric? by jeffmeden · · Score: 2

      The first mistake is that they are still talking in inches instead of metric units.

      This was my thought exactly. If we continue to build new standards around obsolete measurement systems we are just pandering to the Luddites. It is time to move America forward into the 19th century. If we can't engineer for the 21st century, we should turn the creation of standards over to people can.

      Actually most server and rack manufacturers offer specs primarily in cm/mm, and include inches as subtext or a footnote. Upon closer inspection, the Opencompute stuff also offers all drawings this way in mm and in. I suspect they said "this will be 21 inches instead of 19" so it would be more generally consumable, since not everyone finds it as illustrative when they hear "this will be 538mm instead of 482mm". If you can't be bothered to look, then enjoy your time in the 18th century. The rest of us (even if we are in the US) are getting along just fine using the metric system.

    11. Re:metric? by dj245 · · Score: 2

      But units wouldn't need to be labeled if everybody used the same system. The continued existence of the zombie Imperial system is the root cause of the problem.

      No, you should always label your units. My company's (Japanese-based) engineering document system usually lists material stock dimensions in mm. Except for the times that it lists it in meters. And sometimes it is ambiguous where the decimal point should be since that isn't listed (and we often work in 100ths of a mm). There is plenty of confusion because some idiot Japanese designer didn't list his units, and imperial units aren't involved.

      --
      Even those who arrange and design shrubberies are under considerable economic stress at this period in history.
    12. Re:metric? by Jamu · · Score: 2

      The inch is a metric unit. As a consequence of the yard being defined as 0.9144 meters, the inch is exactly 25.4 mm. The Open Rack server slots will be 5.334 meters. Maybe they should round down to 5.333... meters.

      --
      Who ordered that?
    13. Re:metric? by ExploHD · · Score: 2

      Better to just call them 5334 mm racks.

      I preffer 1337 mm racks.

    14. Re:metric? by AmiMoJo · · Score: 2

      What happens when you have a one meter (M) whole unit, that you need to divide into 1/3?

      What happens when you need to divide 1ft into 1/5th? You can cherry-pick random examples of computational inconvenience all day, but metric still has two advantages. Firstly it is easy to plug into a calculator or spreadsheet, making all computation easier. Secondly if you are designing something you can simply use 1200mm instead of 12 inches to get the same divisions in addition to the ease of decimal.

      --
      const int one = 65536; (Silvermoon, Texture.cs)
      SJW, n: "Someone I don't like, and by the way I'm a fuckwit" - AC
    15. Re:metric? by Shompol · · Score: 3, Insightful

      Secondly SI isn't always the best unit of measurement for performing calculations.

      While Imperial units are always the worst.

    16. Re:metric? by Shompol · · Score: 2

      Why would humans care whether water is freezing?

      When the water is frozen, you

      • cannot drink
      • cannot swim
      • not supposed to lick the flagpole
      • cannot get wet
      • cannot get attacked by a crocodile
      • should not eat the fluffy white powder on the ground when it has yellow tint

      It is also essential to set the water temperature below 0 degrees C before adding it to your margarita. So you see, Celsius is more convenient even if you live in exotic countries like Florida!

  3. spinal tap hardware by Hognoxious · · Score: 2

    These are better. They're 2 batter.

    I don't see the point, to be honest. And aren't things supposed to get smaller as technology advances?

    --
    Confucius say, "Find worm in apple - bad. Find half a worm - worse."
  4. 2" shim market by RichMan · · Score: 5, Insightful

    I detect a few years of market window for rails with 1" shims attached (19+2x 1" = 21) to allow old servers to wedge into the new racks.

    1. Re:2" shim market by MarkGriz · · Score: 2

      Or maybe Facebook becomes the new Myspace before this gains any traction, and we can avoid this altoghether.

      --
      Beauty is in the eye of the beerholder.
  5. Server width is changing Rack Width isn't by darthcamaro · · Score: 4, Informative

    There is a big distinction that you need to understand here, the Width of the Server chassis is changing, not the width of the rack itself. The outer dimension of the Server Rack is staying at 24 inches. The REAL problem was a bogus amount of extra cruft in the rack design that is going to be eliminated to make way for the wider servers.

    1. Re:Server width is changing Rack Width isn't by IGnatius+T+Foobar · · Score: 5, Informative

      That "extra cruft in rack design" is where your cables go. I have an 18,000 sqft data center over here and I can tell you from experience that what you call "cruft" isn't nearly enough space for all of the cables once things start getting dense. We are actually considering 23" (telecom standard) racks with 19" rails in them for cabinets that aggregate the networking gear, just for this reason.

      But oh no, far be it from Facebook to actually work with the industry. They screwed up the Internet and now they're going to screw up your data center.

      --
      Tired of FB/Google censorship? Visit UNCENSORED!
  6. already have 23" by Anonymous Coward · · Score: 2, Interesting

    There's already a standard for 23" racks widely used in telecom. So now we have to deal with 19, 21, and 23 options? Great.

  7. Why invent a new standard? by Anonymous Coward · · Score: 5, Insightful

    Why invent yet another new standard when there's a perfectly good one already in use, with lots of inventory in place?

    Telecom has used 23" racks for years. There are standard adapters already available to mount 19" hardware in a 23" rack. 23" racks are already available in the marketplace.

    Further, why 12Vdc? Telecom has been using DC plant for decades and there is a ton of existing 48Vdc equipment on the market. There area existing cabling standards for all this as well.

    Just seems like an attempt to fracture the market and create an opportunity to solve a problem that's already been solved.

    1. Re:Why invent a new standard? by Anonymous Coward · · Score: 2, Informative

      12V DC is a great idea. Eliminate the power supplies inside each individual server and run the entire data center off one massive PSU.

      No, it's a stupid idea thought up by people who are ignorant in matters of distributing electricity. When you reduce voltage you increase amperage, there's no free lunch. Running bus bars around rated in the thousands of amps and up to distribute 12VDC is ridiculously stupid.

    2. Re:Why invent a new standard? by Glendale2x · · Score: 2

      Because they called this one "open" and used a lot of green colors to imply its "greenness".

      --
      this is my sig
    3. Re:Why invent a new standard? by rickb928 · · Score: 3, Insightful

      It's not about the resistive losses, mate.

      First, eliminating power supplies at each server eliminates multiple point heat sources. One honking rectifier stack somewhere does the job. Cool that separately, and maybe even more efficient conversions, though the jury is still out on that for me. Honking-er bus bars will not be as efficient as AC distribution, but the losses will be tolerable. Remember the stack? Getting that cooled outside the server room is useful.

      Using 48VDC makes a lot of sense just because there is a pile of telco equipment made to do that, and well understood. It will have to be regulated at the server level anyways, and the current goes down also.

      But then again, it's not about being compatible, nor even evolutionary. This is a revolution, and if the old guard can't be enslaved, it must be killed off. So telco standards are out of the question. Besides, those extra 2 inches.

      --
      deleting the extra space after periods so i can stay relevant, yeah.
    4. Re:Why invent a new standard? by Bill_the_Engineer · · Score: 2

      You need the higher voltage supply to make up for voltage drop due to resistance found in the power cable themselves. The longer the cable the larger the resistance and the more pronounced the voltage drop. You could compensate by increasing amperage (to make up for lost power) and using heavier gauge cables (to reduce the internal resistance and handle the increased amperage), but the accepted practice is to distribute a higher voltage to the equipment and use a DC-DC converter in the device or immediately at its power input to deliver the required voltage. This is because it's generally safer and cheaper (material costs, weight) to distribute higher voltage than higher current.

      This is why telcos use 48Vdc and the aircraft industry (MIL-STD-704F) use 24 Vdc.

      --
      These comments are my own and do not necessarily reflect the views or opinions of my employer or colleagues...
    5. Re:Why invent a new standard? by petermgreen · · Score: 2

      Telecom has used 23" racks for years. There are standard adapters already available to mount 19" hardware in a 23" rack. 23" racks are already available in the marketplace.

      AIUI they don't want to increase the overall size of the rack so they can fit into standard datacenter floorplans, presumablly this limits the ammount they can widen the server space in the rack without widening the rack itself.

      Telecom has been using DC plant for decades and there is a ton of existing 48Vdc equipment on the market.

      48V DC was well suited to the needs of telcos at the time who originally used it to directly power POTs lines, that doesn't mean it's appropriate for powering computers.

      To prevent currents flowing through metalwork causing ground potential shifts and to limit damage in the event of faults you generally want your computing gear to be isolated from your building power distribution system. This generally implies having an isolating PSU (whether AC-DC or DC-DC) between your building power distribution and your motherboards.

      Afaict this 12V system is about connecting multiple servers in a rack to a common rack level PSU, not about changing building power distribution standards. Presumablly 12V was chosen beause it is becoming the standard for powering motherboards (5V and 3.3V are mostly legacy supplies nowadays)

      --
      note: i'm known as plugwash most places but i screwd up registering that here somehow in the past and now can't register
    6. Re:Why invent a new standard? by tlhIngan · · Score: 2

      Close, Each bus bar pair can support up to 500A, which can be increased by adding more copper. Each bus bar pair can support up to 6KW, while power can be increased by using more copper. The bus bars can be adjusted for higher current, depending on the desired power per column, as they are interchangeable. Normally the Open Rack has three bus bar pairs installed, but it is also configurable with two or one bus bar pairs in the power zone: link

      That bus bar had better be pretty damn big cross-sectionally. 500A is a LOT of current. And if there's three, that's 750A. You better put that power supply at the bottom of the rack, because the IIR losses are huge (they grow with the square of the current - pass twice as much current, and the loss goes up 4 times).

      Pretty much the only way to have racks of it is to make the roof and floor out of solid metal and weld the racks to both sides (IIR losses are huge at connection points). Hell, that's how welding works - high currents, low voltage. Any bit of dirt between the rack's bus bars and the power cable will weld them together over time, if not cause a fire due to heat.

      Hell, the voltage drops are pretty big as well. 500A, if the connection points add 0.01 ohms, you're looking at 5V. (And using IIR, that's a good 250W of power lost). Get it under 0.001 ohms and you're looking at 25W dissipation in the connectors and still half a volt of drop, which is nearly 5% voltage drop.

      There's a reason why telecoms use 48V, and why electrical towers run at hundreds of kilovolts. It's the best mix of voltage conversions and acceptable power losses. Or why datacenters have other voltages like 208V or 240V readily available - lower the current and the cables can be made much smaller.

      Heck, even *CARS* are considering switching to a 48V system to be able to power everything.

    7. Re:Why invent a new standard? by rev0lt · · Score: 3

      Using 12VDC on any type of system that uses more than a few hundred watts is insane

      Like in a car?

    8. Re:Why invent a new standard? by afidel · · Score: 2

      And that increased capital cost will be completely and utterly dominated by the increased efficiency in power useage. If you think these guys haven't analyzed the numbers every which way and run a pilot datacenter to prove out their numbers before releasing this stuff you're delusional. These are not stupid people and they know exactly the problem they are trying to solve. It might not be the same problem as everyone else in the computer sector is trying to solve, but it is a problem quite a few very large web companies are trying to solve (ie this is a solution for Google, Microsoft, Facebook, Amazon, and a few other very large companies with hundreds of thousands to over a million servers).

      --
      There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
    9. Re:Why invent a new standard? by marcosdumay · · Score: 3, Informative

      Touching the 12Vdc @ 500A copper bar bus will kill off some of the new guard too.

      How? Are you trying to tell me that those 500A will run inside a person's body when you apply 12V of potential diference on it? You mean a person's body has 24m Ohm of total resistance?

      The 48V bus would kill much more people, even if less current flowed through it. A person only needs a few mA to die.

    10. Re:Why invent a new standard? by Grishnakh · · Score: 3, Interesting

      They should just dump the -12V requirement. They've had chips available for years that can supply +/-12V for RS232 ports; I believe the MAX232 from Maxim is one of them: you connect it to 5V data lines from your microcontroller, give it 5V for power, add in four capacitors, and it outputs +/-12V RS232 signals. I was using this chip back in the late 90s for embedded equipment that needed RS232 connectivity so we wouldn't need a power supply with -12V.

      They should also dump the 3.3V requirement. AFAIK, that's only used for SATA hard drives, and even there most don't use it because they couldn't count on people actually having 3.3V-compliant power supplies, as many users were installing the (then-new) SATA drives into older computers and using Molex-to-SATA power adapters. 3.3V has a huge drop over any distance, so it makes much more sense to just use a DC-to-DC convertor at the point of use to convert 12V to 3.3V. Now that even CPUs want 12V instead of 5V, and high-end video cards have their own 12V connectors, it seems to me they should just dump all the other requirements altogether, and make a PC power supply only produce +12V, and have the motherboard and drives use their own DC-to-DC converters to produce whatever voltages they require.

  8. Ok, then. by Impy+the+Impiuos+Imp · · Score: 4, Funny

    > Open Compute Developing Wider Rack Standard

    Well, programmers have been getting fatter over the years. I suppose this applies to the female ones, too.

    --
    (-1: Post disagrees with my already-settled worldview) is not a valid mod option.
  9. Re:Wow. Awesome advances in technology! by networkBoy · · Score: 3, Insightful

    As soneone responsible for machine infrastructure, this is a big deal.
    21" racks means that I have to change the layout of my lab. Currently we have the mandatory 42" aisle (or whatever ADA requires) running the length of the lab, with banks of racks pedistileing (sp?) out from the wall with power and data. If all the racks were 2" wider I'd lose a rack in each row, so unless these machines improve density... it's a no sell for me.

    --
    whois gawk date unzip strip find touch finger mount join nice man top fsck grep eject more yes exit umount sleep dump
  10. OK if it's an actual standard by Urban+Garlic · · Score: 2

    All will be forgiven if this standard is better than the current scheme. The *only* thing that's consistent about it is the width.

    The current scheme has a lot of problems with rail-kits fitting into some racks and not others, because they're too deep, or not deep enough, or because one rack has small threaded holes on the inside of the posts, and another has the big square ones. In my set-up, we only have five racks, and already we're running into problems placing equipment because of differences in the mounting geometry.

    Actually, I don't even really need standard mounts, I'd settle for consistent nomenclature -- then at least I could buy adapters, and finally be able to put any piece of equipment in any rack.

    --
    2*3*3*3*3*11*251
  11. 12 Volts? by AB3A · · Score: 2

    Why 12 volts? Telephone companies figured out decades ago that 48 volt positive ground systems were more desirable. They reduced the need for heavier copper wire, and they are not likely to be lethal shock hazards (though burns are certainly possible).

    Furthermore, every motherboard has multiple switching supplies built in. We have 12 volts, 5 volts, 3.3 volts, 1.8 volts, and probably some adjustable voltages too. Some even have separate regulators for individual parts of the board. We will never be rid of the power supplies. We have simply moved them closer to the processors, memory, I/O, and GPU. Why not design the boards to use -48 volt battery systems as primary inputs so that we can reduce corrosion, use existing infrastructure designs, and keep I^2R losses down?

    --
    Nearly fifty percent of all graduates come from the bottom half of the class!
  12. Oldest standard in computing. by Animats · · Score: 2

    19 inch racks are the oldest standard in computing. The ENIAC used 19 inch racks.

    This is really yet another "blade server" scheme. The whole rack is one chassis. Units are on vertically mounted boards with front faces and handles which look like an extra-deep variation on the old Eurocard form factor. That's reasonable enough. It's a lot like 1980s IBM mainframe or 5ESS packaging. Vertically mounted boards are better for airflow, anyway.

    It's not about racks which take horizontal boxes like 19 inch rack components, but are slightly wider.

  13. RTF spec. - benefits and potential issues. by gstrickler · · Score: 4, Informative

    The spec can be downloaded from here

    At first, I thought this sounded like a stupid idea too. Then, I read the spec. They're not just changing the width of the equipment area, and it's not just an extra 2".

    External width is unchanged from the 19" rack standard, it's still nominally 23.6". No replacement of floor tiles or room redesign necessary.

    Equipment width is increased from ~17" (on a 19" rack) to 21", it's allows 4" wider equip.

    Power is handled in 3 "zones" per tower. Each power zone provides 12.5V DC power on each of 3 independent pairs of power rails, No AC power supply is required for each piece of equipment, but they will need DC-DC converters and VRs to supply the voltages needed for their specific components. That saves some space on each device, and provides a slight improvement in efficiency. Because this is standardized worldwide, there is no need for each device to have different power circuitry for different countries.

    Because a zone can have triple power rails, devices can use 1, 2, or 3 power rails to provide whatever level of redundancy is appropriate.

    Space for switches is included in each rack, along with power monitoring/reporting circuitry per rack.

    Battery backup power can either be built into the power supply for each zone, or supplied from a separate battery rack.

    The specification allows for many AC or DC power sources, this is the only significant part of the spec that will vary by country as the power units will need to support the available AC and/or DC supply.

    All devices are to be hot-plug compatible.

    So, it does have a lot of advantages.

    Here are the concerns I have with it:

    All power rails appear to be exposed. While they are on the back, this could be a significant safety (personnel and/or fire) issue. Considering that you can up to 500A @ 12.5V DC running through the zone power rails, and potentially more for the main cabinet DC power rails, exposed seems like a bad idea.

    The standard allows depths from 36" to 48". With the way devices connect to power rails on the back, it looks like this means you will need to use devices designed for the specific depth of your rack, or use a shim to extend the device to match the depth of the rack. I believe they should standardize on one, or at most 3 depths, and have a standard set of shims to connect the devices designed for the shorter depths to fit the deeper racks.

    --
    make imaginary.friends COUNT=100 VISIBLE=false
  14. Re:Wow. Awesome advances in technology! by Doc+Hopper · · Score: 4, Informative

    That's covered in the article. 21" racks within the current 24" standard rack size. By eliminating cable mass in favor of bus-bars, you gain the two inches "free" in your rack footprint. If the rack's designed right, you could even keep the same seismic bracing.

  15. Re:Great news! by Iniamyen · · Score: 2

    Especially when they're STACKED!

  16. The numbers all go to twenty-one by Botia · · Score: 2

    Well, it's two wider, isn't it? It's not nineteen. You see, most blokes, you know, will be serving at nineteen. You're on nineteen here, all the way up, all the way up, all the way up, you're on nineteen on your rack. Where can you go from there? Where?