Slashdot Mirror
Supercomputers - Does the Cabling Matter?
papaia asks: "Having watched, for a while, the development in the area of high-density server hardware solutions (i.e. blade servers), like IBM's 'top gun', and their increased presence in Data Centers, I have been wondering if anybody has had any experience (thus comments) in regards to how important - in such highly priced solutions - is (or could be) the [always neglected] cabling, connecting the servers. One such comment caught my attention, in this regard. Slashdot, how important is the server cabling infrastructure in your Data Centers, and how do you resolve the cable management aspect of it?"
When you're running a large linux cluster, it is absolutely vital to plan your cabling and get it right. It just is, for access, fault troubleshooting, avoiding restricting airflow. Get cable tray, lay it to each rack site (you have planned out banks of rack sites and planned the airflow to your air con, right? - and please, tell me you have air con???).
Allow for larger racks so there's room for cables and PDUs. Really.
Keep things neat with a raised floor, cable tray underneath. A second tray overhead can be handy, but can interfere with moving equipment, so don't fit it unless you'll need it.
Be aware high speed cables, particularly fibre, have a minimum bend radius - go beyond it, and you have fucked up the cable. Also, be aware of sag due to weight of cables - compress the cable too much with the weight of the bundle against cable ties, and you can damage it.
Crosstalk is seldom an issue these days, but be wary of laying power and network too close - even if there is negligible interference, you're safer if they're separated by a decent amount.
Patch panels are useful - use them. Run cables to patch panels and patch panels ->switches, don't go machine -> switch directly (unless you're doing really high-end stuff (for 1GigE copper or fibre, patch panels are fine).