Slashdot Mirror
Supercomputers - Does the Cabling Matter?
papaia asks: "Having watched, for a while, the development in the area of high-density server hardware solutions (i.e. blade servers), like IBM's 'top gun', and their increased presence in Data Centers, I have been wondering if anybody has had any experience (thus comments) in regards to how important - in such highly priced solutions - is (or could be) the [always neglected] cabling, connecting the servers. One such comment caught my attention, in this regard. Slashdot, how important is the server cabling infrastructure in your Data Centers, and how do you resolve the cable management aspect of it?"
When you're running a large linux cluster, it is absolutely vital to plan your cabling and get it right. It just is, for access, fault troubleshooting, avoiding restricting airflow. Get cable tray, lay it to each rack site (you have planned out banks of rack sites and planned the airflow to your air con, right? - and please, tell me you have air con???).
Allow for larger racks so there's room for cables and PDUs. Really.
Keep things neat with a raised floor, cable tray underneath. A second tray overhead can be handy, but can interfere with moving equipment, so don't fit it unless you'll need it.
Be aware high speed cables, particularly fibre, have a minimum bend radius - go beyond it, and you have fucked up the cable. Also, be aware of sag due to weight of cables - compress the cable too much with the weight of the bundle against cable ties, and you can damage it.
Crosstalk is seldom an issue these days, but be wary of laying power and network too close - even if there is negligible interference, you're safer if they're separated by a decent amount.
Patch panels are useful - use them. Run cables to patch panels and patch panels ->switches, don't go machine -> switch directly (unless you're doing really high-end stuff (for 1GigE copper or fibre, patch panels are fine).
Literally speaking the cabling won't matter at all. Whether the resistance of your wires is high or low the electrons are going to travel through it at the same rate. What may be a worry is freak occurances of inductance between wires which could possibly mess with your data, but I'm not sure how common that is. The signal is digital so it's going to be either a 1 or a 0 depending on the voltage of the line, and it's usually difficult to make the voltage do something as drastic as go from +5 to 0 or +5 to -5.
I always have to laugh anytime I go to the store and look at the number of things that say digital nowadays. Wal-Mart sells digital telephone cable for your computer to connect to the wall as if 5 feet of high quality cabling with gold plated tips is going to make a difference in the 30-odd mile trip to your ISP. A friend of mine boasted his "digital" headphones to me once and I had to beat him down to the opinion that "there must be something better about them, so I'm happy" (this sort of pacifist optimism is the bane of IT in general).
Best Buy sells Monster digital audio cables at something like $20 for a 5-ft cable. I had to argue with my father trying to convince him that the cheap RCA cables we already had back at home would be perfectly capable of communicated a digital signal the 5 inches between the DVD player and the receiver. I could have ripped two wires from a speaker cable to connect the two devices and would have gotten just as good sound.
People don't seem to want to realize that digital implies lossless or error-corrected. They don't understand that the "premium quality sound" transmitted between devices can be done using the cheapest electronics equipment available.
I'd save a fortune on car stereo...
Direct away from face when opening.
In any setting the quality of cabling does matter. Or, more specifically, the quality of the connectors on the cable, and the quality of their installation.
Anyone who argues otherwise should recall that the first step in troubleshooting is almost invariably to check the cables.
While I am happy to use zip cord to wire my stereo speakers, I wouldn't trust dollar store cables for anything mission critical.
More important though is to document your cable runs, or even better tag each cable so that you have some idea where it begins and ends. You may know what goes into and out of that big ball of CAT-5 on the floor, but the guy who follows you will have no idea.
Three Squirrels
People don't seem to want to realize that digital implies lossless or error-corrected. They don't understand that the "premium quality sound" transmitted between devices can be done using the cheapest electronics equipment available.
Digital, maybe, but you don't want to raise the ire of the analog stereophiles: You'll get everything from Stereo cables make a difference to Debunking the Myth of Speaker Cable Resonance, not to mention forests worth of dead tree sacrifices for Speaker Cable Face Offs.
And please, please, please, please: Don't get them started on Solid State -vs- Vacuum Tube...