Slashdot Mirror


$600k Fine Over Data Center Death (datacenterdynamics.com)

judgecorp writes: UK contractors Balfour Beatty and Norland have been fined £380,000 ($580k) after an electrician was electrocuted while working on a data center owned by finance firm Morgan Stanley. The fine follows mounting concern that safety is being compromised because of the need for data centers to remain online non-stop. This leads to pressure for contractors to work on live power supplies.

169 comments

  1. 600k? by Anonymous Coward · · Score: 0

    600k is nary a blip on MS' radar. That's not a punishment.

    1. Re:600k? by Anonymous Coward · · Score: 1

      Yeah, a more apt punishment could have been to force management to touch a 300V power line.

    2. Re:600k? by Anonymous Coward · · Score: 0

      Want a Youtube of that

    3. Re:600k? by Z00L00K · · Score: 1

      Should be a 600kV line. That would be more stunning.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    4. Re:600k? by JustAnotherOldGuy · · Score: 1

      600k is nary a blip on MS' radar. That's not a punishment.

      Agreed, it's not even a rounding error to those guys.

      --
      Just cruising through this digital world at 33 1/3 rpm...
    5. Re:600k? by Anonymous Coward · · Score: 0

      More to the point: they would incur a much larger loss with even a brief downtime, so the incentive is to send in the techs to do more life-threatening repairs, even if they are guaranteed to lose cases like this afterwards.

    6. Re:600k? by The-Ixian · · Score: 3, Informative

      Except that I am sure that MS is not on the hook for this at all.

      That's WHY they use contractors.

      Same thing with cell tower owners like Verizon... they own the them but they contract out all the work on them and are not liable for any accidents...

      --
      My eyes reflect the stars and a smile lights up my face.
    7. Re:600k? by sh00z · · Score: 1

      600k is nary a blip on MS' radar. That's not a punishment.

      Yes, but if TFA is correct and the person who died was not informed of the live circuit status, you can add two zeroes to the end of that number when the wrongful death lawsuit is filed.

    8. Re:600k? by Existential+Wombat · · Score: 1

      Yes, but if TFA is correct and the person who died was not informed of the live circuit status, you can add two zeroes to the end of that number when the wrongful death lawsuit is filed.

      This was not in the USA, so those kind of damages won’t be given out.

    9. Re: 600k? by cthulhu11 · · Score: 1

      One would think that the first day of Electrician 101 covers the idea of checking vs assuming.

  2. Nothing Wrong with Non-Stop Service! by Anonymous Coward · · Score: 0

    Didn't RTFA but if companies want 24/7 99.9999% up-time. Then they better have paid for the all the stuff to do it safely including line techs trained and certificated for live work. If they cheaped-out they deserve the an even bigger fine. Yes, live work happens and sometimes for no good reason then trying to save a few bucks. But even power companies do live work on their critical infrastructure and even with best tools and training a life is lost here and there.

    1. Re:Nothing Wrong with Non-Stop Service! by Anonymous Coward · · Score: 2, Insightful

      Isn't it sad that data center downtime is far more expensive than permanent downtime for a human being? This is just absolutely ridiculous and unjust. Someone needs to go to jail for this type of negligence.

    2. Re:Nothing Wrong with Non-Stop Service! by Anonymous Coward · · Score: 2, Insightful

      Didn't RTFA but if companies want 24/7 99.9999% up-time. Then they better have paid for the all the stuff to do it safely including line techs trained and certificated for live work. If they cheaped-out they deserve the an even bigger fine. Yes, live work happens and sometimes for no good reason then trying to save a few bucks. But even power companies do live work on their critical infrastructure and even with best tools and training a life is lost here and there.

      In my experience, the most likely person to pull a dangerous stunt like working on a live high-voltage feed is someone who's got all the certs and experience in the world and is working on top-flight gear - thus meeting your "paid for the all the stuff to do it safely" requirement - but they get careless "just this once" for whatever reason.

      Why?

      Because they're certain they know what they're doing, and they're certain the equipment is safe.

      But they don't, and it isn't.

    3. Re:Nothing Wrong with Non-Stop Service! by Pinky's+Brain · · Score: 1

      Even if they don't know what they're doing, they are at least getting hazard pay.

    4. Re:Nothing Wrong with Non-Stop Service! by Z00L00K · · Score: 2

      The five nine uptime is not counting planned stops. So you can at a datacenter have a planned stop for a month and still conform to the contract. But it wouldn't make sense.

      There is a reason why clustered systems are used - one node goes down another takes over. That's good enough to provide decent uptimes in most cases.

      But today with virtual servers it's often one huge single server, and that's a single point of failure system even if the server itself may have built in redundancy there are always something that can fail.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    5. Re:Nothing Wrong with Non-Stop Service! by stooo · · Score: 1

      HFT Kills.
      Yes, it's sad.

      --
      aaaaaaa
    6. Re:Nothing Wrong with Non-Stop Service! by aaarrrgggh · · Score: 2

      Infrastructure availability should be 4-5 9's for a tier IV facility-- planned and unplanned downtime. Unfortunately, the project in question appears to be a 2N upgrade, which tend to be the most risky projects if done online. A Tier III or Tier II system cannot be safely upgraded online, especially at 400V. It is marginally more practical to do at 208V, but proper safety procedures are essential. You use insulating blankets to safe off any live parts, gear up in the space suits, etc... and it can be done.

      In the US though the fines would be huge. It will eventually lead to either IT failover solutions to remote sites that are 100% reliable, or 3N distribution systems.

      Today though everybody thinks they can design/build/operate a data center. On the last note, I know JLL does a better job at mission-critical than CBRE, and they charge more because of it. The added value of designers today is almost nill-- everybody thinks it is easier than it is because it has become a commodity.

    7. Re:Nothing Wrong with Non-Stop Service! by Ol+Olsoc · · Score: 4, Funny

      Isn't it sad that data center downtime is far more expensive than permanent downtime for a human being?

      What's sad about it? Downtime creates downtime for other human lives too. At some point, you have to acknowledge that this is a trade off, a person assumes risk to their own lives in order to make other peoples' lives better or more productive.

      The mideast called - the said you were the ideal candidate for their new landmine detection program.

      --
      The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
    8. Re:Nothing Wrong with Non-Stop Service! by khallow · · Score: 0, Troll

      Oh, a funny. You do realize that there are real people who actually do risk their lives digging up landmines right? And they do so to save other people from dying or getting maimed by these landmines?

    9. Re:Nothing Wrong with Non-Stop Service! by Coren22 · · Score: 1

      What I find sad is that shutting down one of the power sources would cause downtime, who designs a data center with a single power network?

      There are power line load balancers for hardware that doesn't allow multiple power input, but most equipment allows for multiple power sources to decrease downtime. Why would this guy need to work on a live circuit?

      --
      APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
    10. Re:Nothing Wrong with Non-Stop Service! by Coren22 · · Score: 1

      Not working directly in a data center, what the hell is 2N or 3N?

      I am wondering how the hell a datacenter like this could possibly not have redundant power to allow for one of the power sources to go down for maintenance.

      --
      APK likes to ask for responses to the same things over and over. Maybe he just likes the responses?
    11. Re:Nothing Wrong with Non-Stop Service! by Anonymous Coward · · Score: 0

      Sure. You first.

    12. Re:Nothing Wrong with Non-Stop Service! by Anonymous Coward · · Score: 1

      making middle-class living while risking your life for the lives and safety of others is at least altruistic and noble.

      making middle-class living while risking your life for maximum profits of others is pretty grotesque, and should be illegal.

    13. Re:Nothing Wrong with Non-Stop Service! by khallow · · Score: 1

      making middle-class living while risking your life for maximum profits of others is pretty grotesque, and should be illegal.

      Point to someone who's doing that and we'll see if you know what you're talking about.

    14. Re:Nothing Wrong with Non-Stop Service! by Anonymous Coward · · Score: 0

      2N and similar terminology is a measure of redundancy in systems, where N is the minimum number of "things" needed for operation.

      So, if you are building a datacenter and need generator support, you might choose to purchase N+1 generators. This allows for a single generator failure (or unavailability due to maintenance). Similar schemes can be used for power supplies, UPS systems, etc.

      Lower tier datacenters may be adequately managed with N+1 generators and UPS systems. This allows for a single failure, where the unlikely event of a failure during maintenance may be acceptable.

      2N indicates that an entire power system is replicated. This allows failure of an entire bank of load-sharing generators/UPS systems, and the loads will fail over onto the replicated system.

      2(N+1) is specified for so called "tier IV" datacenters. This provides redundancy, even if an entire system is unavailable for maintenance or due to fault, so permitting continued operation even under multiple fault conditions.

      While dedicate datacenters often have redundant power, this is often not the case with commercial in-house datacenters. For example, my local hospital has an in-house datacenter supporting all its IT systems, and it also provides hosting services for a number of affiliated hospitals in the region. That datacenter has a single mains supply, single generator and a single UPS system - i.e. there is no redundancy. The hospital needed to have some maintenance done on the generator, and a portable generator on a flatbed had to be brought in to cover the maintenance period. That sort of thing is surprisingly common.

    15. Re:Nothing Wrong with Non-Stop Service! by Anonymous Coward · · Score: 1

      We risk our lives for stupid things all the time. It's a matter of how big the risk is. I got in my car and went to my local coffee shop to grab a cup of coffee earlier today. I risked my life by getting into a steel box with wheels and going out on the road so I could get out of the house and enjoy some coffee. But the risk was known and I found it to be worthwhile.

      The problem here isn't that somebody was doing risky electrical work for money. Electrical work is dangerous by its nature, and electrical work for commercial offices is for somebody else's profit by its nature. There's nothing wrong with that on its own. The industry knows what the best practices for safety are, and they know how much risk you're exposed to when you follow those best practices (and it's *never* zero). The problem here was a company not following those best practices and taking on added risk.

    16. Re:Nothing Wrong with Non-Stop Service! by Ol+Olsoc · · Score: 2

      Isn't it sad that data center downtime is far more expensive than permanent downtime for a human being?

      What's sad about it? Downtime creates downtime for other human lives too. At some point, you have to acknowledge that this is a trade off, a person assumes risk to their own lives in order to make other peoples' lives better or more productive.

      The mideast called - the said you were the ideal candidate for their new landmine detection program.

      Because what I wrote was an appropriate if mild response to what you wrote, oh brittle and pathological person.

      So tell me, are you willing to have a person die for some database that you find important?

      Your post at top tells us you are perfectly fine with people dying so that other peopple can have some file stored in a dat center. Uptime to you is more important than someone else's life.

      And the better question is - are you willing to cease existance for some file someone else finds important?

      Probably a different answer, isn't it?

      Hence I was letting you off easy, and not saying what I think of you, because your sociopathy Is rather disgusting. Not that you give a damn about anyone else but yourself.

      --
      The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
    17. Re:Nothing Wrong with Non-Stop Service! by Ol+Olsoc · · Score: 2

      making middle-class living while risking your life for maximum profits of others is pretty grotesque, and should be illegal.

      Point to someone who's doing that and we'll see if you know what you're talking about.

      Why do you want them to kick off so you aren't inconvenienced?

      --
      The shepherds did so well protecting the flock that the sheep no longer believed that wolves existed.
    18. Re:Nothing Wrong with Non-Stop Service! by Archfeld · · Score: 1

      Balanced PDU's are very expensive when compared to standard units.

      --
      errr....umm...*whooosh* *whoosh* Is this thing on ?
    19. Re:Nothing Wrong with Non-Stop Service! by khallow · · Score: 0

      Why do you want them to kick off so you aren't inconvenienced?

      Why do you hate the unicorns so much that you must bring in non sequitur fallacies into your arguments?

  3. Really... by Anonymous Coward · · Score: 0

    Seriously... $600,000 fine? For a human life...
    (The company says) deal! We're in! Much cheaper than our SLA
    non-compliance clause!

    CAP = 'suicidal'

    1. Re: Really... by Anonymous Coward · · Score: 0

      How much do you think they should pay? A lot of other people died that day, most of them in their sleep! Who's gonna pay for that? Until we get the Clinton healthcare death squads in office NO ONE IS SAFE! How many more must die in an attempt to keep Hillary's email server going?

    2. Re: Really... by Anonymous Coward · · Score: 1

      You are an idiot and are COMPLETELY missing the point. The amount is not for just any death. The amount is for enforcing/encouraging or even just allowing a work environment that is unsafe just so that the bottom line doesn't get affected.

      That kind of negligence needs to hurt. $600K doesn't even cause a twinge of pain. I am sure the amount of money they were trying to save by not allowing the work to be done safely was a lot more than that.

    3. Re:Really... by v1 · · Score: 4, Informative

      The 600k was the fine for non-compliance. You'd get that whether or not someone was killed. (some fines will get a bumper for injury, but not many have a bump for death for some reason)

      Ret assured, there will be a multi-million dollar lawsuit filed by the family that will get settled out of court for an "undisclosed amount". (around 4 million is par) The fine was just the wakeup-call for the board to find a scapegoat to be the focus of the PR crucification and actual painful monetary loss for the impending lawsuit. The way things like this usually go, if the press doesn't dig up any real pattern of misconduct, there will probably just be someone issuing a public apology. If they do find a pattern, someone will get the axe.

      Unfortunately, these places rarely get a fine unless someone is injured or killed, because nobody knows or cares about the noncomp until it hits the papers. Then the regs look bad if they don't step in and issue a fine like they ought to have done several times in the past to have, y'know, prevented this from happening in the first place.

      But regardless of what happens, hopefully there will be changes made. From the looks of it, the tech that got killed was unaware that the wire that got him was energized, due to poor communication from his management, which appears to have been the result of poor communication from upper management and whoever was coordinating the work with the other group that was in charge of the deadly wire. So it's a bit early to be blaming the tech. Heck, he may have opened the box and tested it and found it wasn't connected yet and was safe to leave open, got to work, connecting it to something else, and half an hour later someone in another building lit the box up and the tech never knew what hit him. Things like that can happen when two different groups are working on connected systems and are unaware of each other and not keeping in communication as shared circuits are cut and energized.

      --
      I work for the Department of Redundancy Department.
    4. Re:Really... by Peil · · Score: 4, Informative

      "Ret assured, there will be a multi-million dollar lawsuit filed by the family that will get settled out of court for an "undisclosed amount". (around 4 million is par)"

      Very unlikely, it was in the UK, we don;t get settlements anywhere near that level

    5. Re: Really... by elal1862 · · Score: 1

      It's even sadder than that: The extra cost of hiring people who were competent enough to recognise and mitigate the risks, would've likely been less than $580k.
      In these harsh economic times, it's really a race to the bottom.

    6. Re:Really... by Anonymous Coward · · Score: 0

      From the looks of it, the tech that got killed was unaware that the wire that got him was energized, due to...

      Not to detract from the other, very real, issues that are being discussed here, but isn't this something that any competent electrician would test for before working on the wire?

    7. Re: Really... by kilfarsnar · · Score: 1

      It's even sadder than that: The extra cost of hiring people who were competent enough to recognise and mitigate the risks, would've likely been less than $580k. In these harsh economic times, it's really a race to the bottom.

      But now the Holy Dollar rules everybody's lives

      Gotta make a million, doesn't matter who dies

      --
      "What the American public doesn't know is what makes them the American public." -Ray Zalinsky (Tommy Boy)
    8. Re:Really... by budgenator · · Score: 2

      Neither do we for the most part, at first the sympathetic jury awards the big-bucks, which is then paid out as say 4 Million over the life of a 30 year structured settlement. Later after the elections are over, and the emotions die down the award is reduced on appeal.

      --
      Apocalypse Cancelled, Sorry, No Ticket Refunds
    9. Re:Really... by whit3 · · Score: 1

      From the looks of it, the tech that got killed was unaware that the wire that got him was energized, due to...

      Not to detract from the other, very real, issues that are being discussed here, but isn't this something that any competent electrician would test for before working on the wire?

      Any competent, licensed, journeyman or master electrician would take responsibility for his safety in a number of ways: often, by disabling power to a box and applying a padlock so no other worker could reenable that power. It isn't clear from the article what the issue was, but the citation that went with the fine indicates a failure of an extended team to communicate. Some of the people involved knew that the box had live wires, while the victim did not, perhaps? Did the victim have his own electrical tester, or was he a semiskilled helper?

    10. Re:Really... by david_thornley · · Score: 1

      Back around 2000, I worked for a company doing power distribution software. One of the EE-type guys there told us that, in Britain, the electrician is supposed to go by what he's told as to whether a line is hot or not. (This was motivation to do our job well, since it kept track of things like lines and what was powered.) In the US, if we screwed up, the electrician was going to take his or her own precautions.

      --
      "When you have eliminated the unacceptable, whatever is left, however improbable, must be the truthiness" - Holmes
    11. Re:Really... by v1 · · Score: 1

      As stated above, my scenario is it was dead when he started, and someone cut power back in during his work. I'm going on the assumption the tech was following good practices and that management/communication was where the fault occurred, based on the background provided in the article.

      But the correct way to address that particular risk is to tie said source wires to ground with heavy cable. That way if some clown does light up the wires while you're working, the worst you're going to get is a bath of copper sparks before a breaker trips somewhere in the line. (unpleasant to be certain, but not lethal)

      If I was going to be working in an environment where I was trusting somoene possibly a long distance away to keep their fingers off the knife-switch while I worked, (where driving to another location to install a lockout may not be practical) I think webbing the lines to ground while working on them is a precaution I would be consistently taking. I've read too many stories about clueless idiots blindly switching power back on when they stumble across a switch or breaker that's off.

      --
      I work for the Department of Redundancy Department.
    12. Re:Really... by cwsumner · · Score: 1

      Back around 2000, I worked for a company doing power distribution software. One of the EE-type guys there told us that, in Britain, the electrician is supposed to go by what he's told as to whether a line is hot or not. ...

      Britain is a conqured country owned by invading generals, and has been since 1066 AD ! 8-)
      Get yourself a set of padlocks and tags!

  4. The fine won't hurt the DC owners. by dwywit · · Score: 3, Insightful

    How about a mandatory downtime for the data centre of say, 24 hours?

    Hit 'em in the hip pocket - which is what a fine is supposed to do, but rarely, in the case of corporations, achieves its desired affect.

    --
    They sentenced me to twenty years of boredom
    1. Re:The fine won't hurt the DC owners. by Ostracus · · Score: 1

      Downtime? That would be when the work doesn't automatically switch to the other data centers, right?

      --
      Shai Schticks:"You don't make peace with friends, you make peace with enemies"
    2. Re: The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      This is genius.

    3. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 2, Insightful

      Someone made the call to have live circuits worked on, despite this being illegal. That person should be tried for manslaughter. It's about time people stopped hiding behind the corporate veil and are made accountable for their decisions, just as you or I would be.

    4. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      Pay $600k fine or build a new data center... wonder what Morgan Stanley's underwriters (or is than an actuary) would say.

    5. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      How about a mandatory downtime for the data centre of say, 24 hours?

      Why? I cut corners because the client was in a hurry is a pisspoor excuse. Clients are always in a hurry.

    6. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 1

      There's a degree of incompetence from the engineer's side too. The first rule of electrical engineering is "turn it off". If it absolutely positively can't be turned off, don a Faraday suit and wear isolating gloves.

    7. Re:The fine won't hurt the DC owners. by dwywit · · Score: 1

      Yep, but an unscheduled 24-hour outage is still going to look bad on *someone's* quarterly report.

      The workload might switch elsewhere, but fixed costs don't.

      --
      They sentenced me to twenty years of boredom
    8. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      It's AC goddammit.

    9. Re:The fine won't hurt the DC owners. by bertoelcon · · Score: 1

      Someone made the call to have live circuits worked on, despite this being illegal. That person should be tried for manslaughter. It's about time people stopped hiding behind the corporate veil and are made accountable for their decisions, just as you or I would be.

      No, companies are people now, better to charge the company with manslaughter and figure out how to send it to prison.

      --
      Anything can be found funny, from a certain point of view.
    10. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      And what Detective Comics or Anonymous Coward have anything to do with it?

    11. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 1

      Just require it to shut down all operations. However, people will still be entitled to their paychecks while the corporation is in prison, just as we still require child support payments from fathers in prison who are unable to work.

      If the corporation can survive being completely shut down for 2-3 years while paying its employees, creditors, lawsuits from breached contracts, etc from savings, then it can resume operations afterwards. I'm not sure many could survive that though, outside of maybe Apple and a handful of others. But the same can be said for most humans who get sent to prison and have their entire lives derailed.

    12. Re:The fine won't hurt the DC owners. by AlecC · · Score: 1

      Reading the article, it appears that it is not that "somebody made the call", it is that communications between the teams working on the project, from two companies, was so bad that one crew didn't realist that people would be working in the area, and the other crew didn't realise it was live. Incompetence, not risk-taking.

      --
      Consciousness is an illusion caused by an excess of self consciousness.
    13. Re:The fine won't hurt the DC owners. by AlecC · · Score: 2

      But to know to turn it off, you have to know it is on. And that appears to be the problem here - people didn't know what was on.

      --
      Consciousness is an illusion caused by an excess of self consciousness.
    14. Re:The fine won't hurt the DC owners. by Lumpy · · Score: 1

      $10X the "fine" must be paid directly to the victims family, Plus a mandatory Data-center full downtime of 48 hours while the "investigation" takes place.

      if they find ANY problems the fines and time down multiply.

      Make a death like this capable of completely crushing the company.

      --
      Do not look at laser with remaining good eye.
    15. Re:The fine won't hurt the DC owners. by PolygamousRanchKid+ · · Score: 1

      Well . . . I was wondering what the dollar value of a human life is these days. So now, I guess that it is about $600K . . . in the UK. The value of human lives might vary in other countries.

      You might want to consider that when planning where to put your data center . . .

      --
      Schroedinger's Brexit: The UK is both in and out of the EU at the same time!
    16. Re:The fine won't hurt the DC owners. by PAjamian · · Score: 1

      The second rule is "treat every circuit as live, even if you know it is not" So yes the electrician was not taking the proper precautions if he was not treating the circuit as live.

      --
      Windows is a bonfire, Linux is the sun. Linux only looks smaller if you lack perspective.
    17. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      But to know to turn it off, you have to know it is on. And that appears to be the problem here - people didn't know what was on.

      Dude, if it's electrical and you don't KNOW that it's off, then it's ON.

    18. Re:The fine won't hurt the DC owners. by PAjamian · · Score: 1

      Yes, and a competent electrician always assumes a circuit is live, even if he knows otherwise. You never know when some idiot is going to flip the switch back on that you turned off to kill the circuit, or if you simply flipped the wrong switch or clear forgot. If you're working on an electrical circuit you *always* assume it is live, no exceptions!

      --
      Windows is a bonfire, Linux is the sun. Linux only looks smaller if you lack perspective.
    19. Re:The fine won't hurt the DC owners. by PAjamian · · Score: 1

      Mandatory downtime would not be punishing the DC, it would be punishing the customers of the DC would would loose far more money from loss of business than the DC itself would.

      --
      Windows is a bonfire, Linux is the sun. Linux only looks smaller if you lack perspective.
    20. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      and such places would normally have to put a pad lock on the cut off switch, so the only guy who can turn the power on is the guy who turned it off and put the lock on

    21. Re:The fine won't hurt the DC owners. by PAjamian · · Score: 1

      ...Which entirely misses the point that you always always always assume a circuit is live when working on or near it.

      --
      Windows is a bonfire, Linux is the sun. Linux only looks smaller if you lack perspective.
    22. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      That's what my teacher used to say:

      If you are sure it is turned off, you have no problem touching the wires.
      If you hesitate, you are not sure and have to asume it is still turned on.

    23. Re:The fine won't hurt the DC owners. by vlad30 · · Score: 1

      Ditto and for $2 you can get a live wire electrical tester $5 will get you a contact-less version

      --
      Your'e all thinking it, I just said it for you
    24. Re: The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      The death of a 99 percenter should warrant that? I think you seriously overestimate your worth, peon.

    25. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      People work on live circuits all the time. The matter is not whether voltage was applied or not, it's whether the workers were competent enough to use precautionary measures that correspond to the level of threat.

      That being said, from a certain voltage level the necessary measures make it impossible to work, so you HAVE to switch if off.

    26. Re:The fine won't hurt the DC owners. by Z00L00K · · Score: 2

      Turn it off, then do a short circuit to ground at the work position as well so if someone turns on the power then the fuse will blow. If they are unlucky then the main fuse will blow and that's going to make a mark in someone's report.

      If it's really bad the UPS will die - and that will definitely make a mark in the budget for that year.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    27. Re:The fine won't hurt the DC owners. by Z00L00K · · Score: 1

      Assuming that there's no asshat turning on the power while you are working because they needed the power NOW.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    28. Re:The fine won't hurt the DC owners. by vlad30 · · Score: 1

      Not illegal everywhere but precautions must be taken when doing live work.

      --
      Your'e all thinking it, I just said it for you
    29. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      Let's call it an even $**2*6000k.

    30. Re:The fine won't hurt the DC owners. by Z00L00K · · Score: 1

      To paraphrase what happens when programs do something bad: "The company has executed an illegal operation and will be terminated".

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    31. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      why? somethings cannot be done live, so you'll have to verify it is not, and prevent some idiot from turning it back on while you are working

    32. Re:The fine won't hurt the DC owners. by jabuzz · · Score: 1

      Live line working is standard industry practice for power distribution.

      https://en.wikipedia.org/wiki/...

      I see no reason why similar procedures cannot be developed for any electrical work.

      In general there is fewer incidents in live line working that dead line working. Mostly because if you know it is live then you will be treating with the utmost respect.

    33. Re: The fine won't hurt the DC owners. by Anonymous Coward · · Score: 1

      The official rules in Switzerland are:

      - turn it off
      - secure against accidently turning it on (e.g. a lock on the switch)
      - test for voltage
      - ground the wires
      - work

      Its a pain in the ass and a lot of workes disrespect the rules.

      But advice like "treat every wire as live" is pretty useless. What do you do with a live wire?

    34. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      And that's why they have rule number 2 - always treat the circuit as live even if you know it isn't.

    35. Re:The fine won't hurt the DC owners. by jabuzz · · Score: 3, Informative

      It happened in the United Kingdom and live line working is perfectly legal. If you read the article you will see the death occurred due to a lack of communication causing the people to be unaware they where working in the vicinity of line equipment.

      Here is a link to the Health and Safety Executive's press release on the subject which has more details. The full judgement does not seem to have hit the judiciary web side yet. At least my searches are coming up blank.

      http://press.hse.gov.uk/2015/e...

      Basically the fines where from not operating proper health and safety systems. If someone has enough patience you should be able to dig out the full judgement from the

    36. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 5, Informative

      This is why (at least in the US), standard (and required) practice is to place a lock on the source of energy, and retain the only key to said lock. If multiple workers are all working on the same circuit, each worker has his (or her) own lock on the circuit.

      It's called lockout/tagout, and there are SEVERE fines for removing somebody else's lock (and if somebody gets killed, due to your removing his lock, that would be considered manslaughter)

    37. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      In that world, only the large companies will survive. Insurance costs alone would bankrupt most small companies, some would die off after their fine and downtime, and the rest because they can't compete if they follow every safety regulation.

    38. Re:The fine won't hurt the DC owners. by shortscruffydave · · Score: 2

      The second rule is "treat every circuit as live, even if you know it is not" So yes the electrician was not taking the proper precautions if he was not treating the circuit as live.

      Coincidentally, last night I was talking to my father who worked as an electrician/electrical engineer for over 30 years. He said that back in the day, it was quite common for sparkies to work on live kit....they knew the cables were live and knew how to respect them and what precautions to take. This wasn't just restricted to the standard 240V supply (c.f. the 110V used on the other side of the Pond) but also to distribution kit, running well into the kilovolt range. For example, bare end of armoured cable going into a substation being wiped and jointed by hand.....carefully, very carefully

    39. Re:The fine won't hurt the DC owners. by sociocapitalist · · Score: 1

      How about a mandatory downtime for the data centre of say, 24 hours?

      Hit 'em in the hip pocket - which is what a fine is supposed to do, but rarely, in the case of corporations, achieves its desired affect.

      Because you'd be punishing the customers, not only the DC.

      --
      blindly antisocialist = antisocial
    40. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      "Someone made the call to have live circuits worked on, despite this being illegal"

      Are you serious? You do know this is not "illegal", and in fact is a specialized field right?

      https://en.wikipedia.org/wiki/Live-line_working

    41. Re:The fine won't hurt the DC owners. by flink · · Score: 2

      You should be able to lock the breaker out and take the key with you so they can't turn the power on.

    42. Re:The fine won't hurt the DC owners. by locofungus · · Score: 1

      Someone made the call to have live circuits worked on, despite this being illegal.

      In the UK - where this happened - it's not illegal to work on live circuits. The HSE has this in the FAQ:

      When is it safe to work on live electrical equipment?

      It is never absolutely safe to work on live electrical equipment. There are few circumstances where it is necessary to work live, and this must only be done after it has been determined that it is unreasonable for the work to be done dead. Even if working live can be justified, many precautions are needed to make sure that the risk is reduced 'so far as is reasonably practicable'. See: Electricity at work: Safe working practices for more details.

      --
      God said, "div D = rho, div B = 0, curl E = -@B/@t, curl H = J + @D/@t," and there was light.
    43. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      The second rule is "treat every circuit as live, even if you know it is not" So yes the electrician was not taking the proper precautions if he was not treating the circuit as live.

      Coincidentally, last night I was talking to my father who worked as an electrician/electrical engineer for over 30 years. He said that back in the day, it was quite common for sparkies to work on live kit....they knew the cables were live and knew how to respect them and what precautions to take. This wasn't just restricted to the standard 240V supply (c.f. the 110V used on the other side of the Pond) but also to distribution kit, running well into the kilovolt range. For example, bare end of armoured cable going into a substation being wiped and jointed by hand.....carefully, very carefully

      And there are very good reasons why it's not done that way anymore.

    44. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      Mandatory downtime would not be punishing the DC, it would be punishing the customers of the DC would would loose far more money from loss of business than the DC itself would.

      Simple, the customers then bill the DC for lost business.

    45. Re:The fine won't hurt the DC owners. by moeinvt · · Score: 1

      That might not be the full value. Note that this was a fine imposed for violations of the "Health and Safety Work Act". Maybe the UK is different, but in the USA, you could be certain that a civil lawsuit was in the works.

    46. Re:The fine won't hurt the DC owners. by i.r.id10t · · Score: 1

      Or a BOFH or his PFY thinking the worker is in fact a member of Management or perhaps Accounting ...

      --
      Don't blame me, I voted for Kodos
    47. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      That's why they have lockouts. Everyone working on the circuit has their own lock which they use to lock the panel off. No asshat can turn the power back on until everyone has removed their locks. (It's usually a breaker box with multiple latches.)

    48. Re:The fine won't hurt the DC owners. by alistair1978 · · Score: 2

      This is why (at least in the US), standard (and required) practice is to place a lock on the source of energy, and retain the only key to said lock. If multiple workers are all working on the same circuit, each worker has his (or her) own lock on the circuit.

      It's called lockout/tagout, and there are SEVERE fines for removing somebody else's lock (and if somebody gets killed, due to your removing his lock, that would be considered manslaughter)

      It's mandatory in the UK too. Often ignored by independent electricians working by themselves, but hopefully less so by those working for large companies....... UK Health and Safety Executive page on the topic: http://www.hse.gov.uk/safemain...

    49. Re:The fine won't hurt the DC owners. by budgenator · · Score: 1

      If they are doing high-frequency trading, the extra milliseconds that the other datacenter is taking will kill their profits.

      --
      Apocalypse Cancelled, Sorry, No Ticket Refunds
    50. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      The downtime would crush the companies clients just as much as them.
      And paying out $6M to the family for a death that can easily happen "accidentally" will make a lot of poor people trying to feed their kids face a serious dilemma.

    51. Re:The fine won't hurt the DC owners. by Locke2005 · · Score: 1

      Standard Operating Procedure is to turn it off at the breaker box, then Red Tag the switches so that nobody turns it back on again. Anybody other than the person that red tagged it turning it back on risks getting their ass kicked for endangering someone else's life. As a matter of general principle, I'd probably throw a meter across the circuit to make sure it's not live before touching anything too, but that's not required, and there should be plenty of indicator lights anyway. I'd love to know what the true story is of how this guy was working around live circuits in the first place. Getting across anything higher than 120v is painful as hell, I've only managed to touch 240v (touched the prongs while plugging it in behind a dryer), and it was NOT pleasant!

      --
      I've abandoned my search for truth; now I'm just looking for some useful delusions.
    52. Re:The fine won't hurt the DC owners. by budgenator · · Score: 1

      Assuming that there's no asshat turning on the power while you are working because they needed the power NOW.

      That should be controlled by appropriate Lockout-Tagout, hard to turn something on when the switch is padlocked and the key is in your pocket.

      --
      Apocalypse Cancelled, Sorry, No Ticket Refunds
    53. Re:The fine won't hurt the DC owners. by Z00L00K · · Score: 1

      Well, it works for the main switch in a computer hall, but often power is only cut off on a circuit breaker for the circuits where the work is done, not the whole data center.

      Unfortunately way too common. Often because the installation shall be cheap.

      --
      If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
    54. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      That's why circuit breaker lockouts are used. When I worked in the trade, it wasn't unusual to have 3-4 locks on a multi-gang lockout device.

    55. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      that's not how it works. dope.

    56. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      You think they only have one? Cute.

    57. Re:The fine won't hurt the DC owners. by angel'o'sphere · · Score: 1

      In your mind that might be easy.

      However how do you actually connect a life power conduct to a machine?

      You don't. You switch the power off.

      So how exactly do you now do the 'I connect it, but assume it is still life' when actually connecting a life power line is impossible?

      Sorry ... some of the posters repeating that myth are: morons.

      --
      Cost free eBook I read (by iBook/Kobo/Amazon/ObookO/Gutenberg etc.): "The Green Odyssey" by Philip Jose Farmer.
    58. Re:The fine won't hurt the DC owners. by RobinH · · Score: 1

      You speak the truth, and yet I've worked with a lot of maintenance people and electricians who just don't lockout/tagout. They all think they're immune, I guess. Most electricians are used to working on live equipment.

      --
      "I have never let my schooling interfere with my education." - Mark Twain
    59. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 1

      The issue in this case was that it was not meant to be live line working. The electrician had been issued a "permit to work" authorizing him to perform work on a switchboard which had been isolated elsewhere.

      This was a complex case where there were 2 different companies overseeing the work. The work was a 2N power upgrade for the datacenter. Company A was a buildings management firm who oversaw general electrical maintenance, and company B was an electrical installation contractor who had been instructed to perform the upgrade. Company B had installed a new power feed to the building.

      A dual-feed switchboard had been installed and connected to both the existing and new supply. A sub-contractor was hired to connect the dual feed switchboard to the building loads. An electrician for the subcontractor contacted company A to ask for the supply to the switchboard be shut off to permit installation of a new cable. Company A did so, and issued a permit to work on the switchboard, indicating that the switchboard was dead. However, company A only had authority for the existing supply, and did not control the new supply which was being managed by company B until completion of the upgrade.

      During the work, the worker accidentally contacted the live B supply. As he was expecting a "dead" switchboard, appropriate precautions and PPE were not used.

    60. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      One day my friend (a true EE) was buying a short piece of welding cable.
      I asked him what he'd use such a short piece for, even though it was triple-0 wire.
      He said to bridge the line near where he was working. In case someone cuts the lock
      (which he says is very common - but it only has to happen to you once), it'll
      prevent them from bringing the circuit live - their breakers will fault immediately.

      He's still alive (he works on train lines in the NE)...

    61. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      It's actually not illegal to work on live circuits. Highly paid guys do it all the time. You just have to have the right certifications. Essentially, they wear these special heavy duty rubber gloves with leather gloves over them. The rubber gloves have to be tested every day for even a pinhole or else they will be electrocuted. The guy who died here was likely just some lowly electricity, not a live circuits guy. They don't turn off the power to cities when a transformer needs to be changed. Also, working on live stuff even with the right equipment is a very very risky job.

    62. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      Are you joking? Even an amateur electrician has an inductive amplifier that lets you "hear" the AC. One of the benefits of AC over DC, actually.

    63. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      This is a bit like saying a restaurant should still be allowed to operate if the rat turds on the floor are nailed down because then they are part of the permanent structure and not the fault of the restaurant.

    64. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      Every sparky's apprentice knows to never trust such devices. The contactless ones are especially unreliable.

    65. Re:The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      Yeah. That's neither how things work, nor is it "simple". SLAs limit damages. Contracts exclude force majeure. And frankly, you're just stupid.

    66. Re:The fine won't hurt the DC owners. by jabuzz · · Score: 1

      Except live line working is still common practice and has lower incidents of fatalities than dead line working.

    67. Re:The fine won't hurt the DC owners. by RockDoctor · · Score: 1

      Assuming that there's no asshat turning on the power while you are working

      This is what lockout-tagout isolators are for. YOU turn off the power on the circuit which YOU are going to be working on, and YOU put a padlock through the isolator locking it isolated ; and that padlock is normally a combination lock which YOU and ONLY you know the number for.

      This is not rocket science. It is designing safety into the system from day one. These days it is actually getting hard to find isolators which don't have the relevant holes for locking out.

      Procedures weren't being followed. That's what is being fined.

      --
      Birds are not dinosaur descendants;birds are dinosaurs, for all useful meanings of "birds", "are" and "dinosaurs"
    68. Re: The fine won't hurt the DC owners. by Anonymous Coward · · Score: 0

      "What do you do with a live wire?" - ground it, of course...

  5. How about IMPRISONING those responsible by Anonymous Coward · · Score: 0

    This was done by individuals, there must be a record of who was in control and made these decisions - they are responsible for manslaughter, as individuals, and should be tried for that crime.

    1. Re:How about IMPRISONING those responsible by AlecC · · Score: 2

      The problem being, apparently, that nobody made "the decision". Due to lack of communication, one crew thought that status was A, and the other that it was B. Should you sent to prison the person who allowed live power in an area he thought there was nobody working, or the person who sent people to work where he thought the power was off? Or the bosses in the two different companies involved? Or the bosses in the employing company, a bank, the only place the chains of command met, who though they were employing competent contractors? The problem is that the structures were so confused that, though they didn't realise it, there was no-one in control. Finding someone guilty "beyond reasonable doubt" is almost certainly impossible.

      --
      Consciousness is an illusion caused by an excess of self consciousness.
    2. Re:How about IMPRISONING those responsible by Pinky's+Brain · · Score: 1

      Both those people had people on site at the top of the totem pole. They might not have realized that no one was in control, but they were certainly paid to be.

      Pull their certs.

    3. Re:How about IMPRISONING those responsible by l0n3s0m3phr34k · · Score: 1

      "never trust always test" would have saved his life.

    4. Re:How about IMPRISONING those responsible by Nidi62 · · Score: 2

      The problem being, apparently, that nobody made "the decision". Due to lack of communication, one crew thought that status was A, and the other that it was B. Should you sent to prison the person who allowed live power in an area he thought there was nobody working, or the person who sent people to work where he thought the power was off?

      Did the company have a lockout procedure? If yes, was the procedure followed and if not, why? If the answer to the first is "no" or the second is "employees weren't trained or improperly trained" then that company is liable for the worker's death.

      --
      The only thing necessary for evil to triumph is for it to be pitted against a slightly greater evil
    5. Re:How about IMPRISONING those responsible by AlecC · · Score: 2

      The company was certainly liable, and has been fined. The question is, was any person liable, to the extent that the could be imprisoned for manslaughter?

      --
      Consciousness is an illusion caused by an excess of self consciousness.
    6. Re:How about IMPRISONING those responsible by Anonymous Coward · · Score: 0

      So the dead guy? I mean, shit, every wire I touch I hit with the field detector first. Even when doing live work, I check and make damn sure it's live, because if it isn't, there's something else that's very wrong.

    7. Re:How about IMPRISONING those responsible by gnupun · · Score: 1

      The problem being, apparently, that nobody made "the decision".

      This is no different than any murder investigation and the police should find out who exactly made the decision and make them pay for it.

    8. Re:How about IMPRISONING those responsible by AlecC · · Score: 1

      I imagine there has been such an investigation. But first, you have to decide what was "the decision" that caused the accident. In aviation they say that an incident is none things going wrong at once, an accident is ten. If five people each believe that one of the other four had put the safety locks in place, and none had, which of the five made the decision which caused the accident?

      --
      Consciousness is an illusion caused by an excess of self consciousness.
    9. Re:How about IMPRISONING those responsible by Anonymous Coward · · Score: 0

      My take is that the real fault depends on the nature of the miscommunications. If the now deceased contractor asked confirmation that the line was cold before starting, and no one in the last-minute check clarified which line was cold, then everyone in that chain of communication is guilty of negligent manslaughter (unless some evidence surfaces that this was a premeditated murder by work hazard). If the electrician attempted no last-minute confirmation, then his own negligence must be recognized as a very significant factor in his death.

    10. Re:How about IMPRISONING those responsible by rubycodez · · Score: 1

      Medium voltage electricians are expected to be able to perform work under such conditions. He fucked up, now he's dead.

    11. Re:How about IMPRISONING those responsible by hey! · · Score: 1

      Not really. First of all he wasn't an electrician, he was something called a cable jointer; it doesn't require a license and normally they don't work on live stuff. Secondly, according to TFA he wasn't aware the line was live, which means someone in a supervisory position screwed up.

      As to whose fault it is, that depends. If the company in charge of the site has a proper lock out/tag out policy and training program, it's probably some supervisor's fault -- unless the worker went into an area specifically against instructions in which case it's the worker's fault. If the company doesn't have a proper LOTO program (including training for all personnel on site) it's the company's fault.

      LOTO may be a PITA, but it's not rocket science.

      --
      Post may contain irony: discontinue use if experiencing mood swings, nausea or elevated blood pressure.
    12. Re:How about IMPRISONING those responsible by ChumpusRex2003 · · Score: 1

      The company did have a lockout procedure and a "permit to work" procedure. The problem was that there were two different companies working on the same site and they failed to have a proper protocol when one needed the other to disconnect power.

      The facility had a single power feed, and the routine maintenance and supervision of the electrical system in the building was outsourced to a building management company. The provision of a second supply and associated switching system was being performed by a specialist electrical contractor.

      The time came to disconnect connect the IT loads from the incumbent single-feed switchboard to the dual-feed switchboard during a period of scheduled downtime. The building management contractor issued a "permit to work" to a cable jointer to disconnect a sub-switchboard from the main supply and connect it to the new dual-feed switchboard; the permit certified that the main supply had been turned off and locked out, and would only be unlocked upon return of the permit.

      The original plan had been to connect the IT loads first, then connect power to the new switchboard. However, because of a specification error when procuring the switchboard, it had been modified on-site, and following modification required live tested prior to connection of any IT equipment. As a result, it had already been connected to both the existing supply (under the management of the facilities management company), and also to the new supply (which had not yet been handed over, and remained under the control of the installing contractor), before any outgoing cables were connected.

      While the building management company had disconnected the building's main supply and locked it off prior to issuing the permit to work on the switchboard, they had failed to contact the contractor handling the 2nd supply and failed to ensure that the 2nd supply was also locked out. As a result, when the cable jointer set to work, the switchboard was still energised by the 2nd supply. Although the connections that the jointer was working on were dead and isolated by a switch in the switchboard, as the switchboard was open, he accidentally contacted the busbars fed from the 2nd supply and was electrocuted.

    13. Re:How about IMPRISONING those responsible by rubycodez · · Score: 1

      oh, so he did an electricians job, not being one, and killed himself

  6. Who took the decision to undertake the work? by Anonymous Coward · · Score: 1

    Working live is recognized as an acceptable risk in circumstances where it is not practicable to turn stuff off (UPS's and backup-generators may have made this a logistics nightmare even if you DGAF about uptime.) There are qualifications you can get to prove your competence to work live, and there are some very comprehensive procedures to follow that make live working a reasonably safe undertaking (even for the boys doing live joints on HT cables)

    If the guy was a "competent person" in the eyes of the law, then he was perfectly within his rights to take the decision and was fully and personally responsible for any consequences of doing so. If he was not a competent person then he had no business attempting the work.

    I'm a UK fully qualified electrician, I know there are plenty of lads who have been "on-the-tools" since they left school and call themselves electricians, but those of us who actually have the relevant bits of paper can enjoy a great deal of professional autonomy and responsibility, and I find this decision insulting.

    1. Re:Who took the decision to undertake the work? by Pinky's+Brain · · Score: 3, Informative

      From reading TFA it seems he didn't know it was live because there were 2 companies at work without any single person being responsible for coordination.

    2. Re:Who took the decision to undertake the work? by Anonymous Coward · · Score: 0

      Then it's his own fault for not locking off a breaker and / or testing it. There are multi-padlock hasps specifically for these situations.

    3. Re:Who took the decision to undertake the work? by elal1862 · · Score: 2
      When I read in the HSE publication that the hapless person "was electrocuted when his forehead made contact with the 415V live terminals of the second unit", I immediately facepalmed. The apparent lack of safety awareness truly boggles my mind.

      If he was not a competent person then he had no business attempting the work.

      The sad truth is, that the incompetent don't realise that they're not competent at the job.

    4. Re:Who took the decision to undertake the work? by Anonymous Coward · · Score: 1

      Exactly. A well managed workplace gives you tags to add to your padlocks with your name, picture, "DO NOT OPERATE", and "MY LIFE IS ON THE LINE".

      The picture is really key. No matter what your boss tells you, seeing the face of the guy you might kill will make you think twice.

    5. Re:Who took the decision to undertake the work? by locofungus · · Score: 2

      From the press release from the HSE he wasn't working on live equipment.

      He was given a work permit by the site operator to route some power but the route was through other equipment that had exposed live connections. The electrician made inadventent contact (with his forehead) with one of these exposed live connections.

      Worse, the site operator was aware of this risk, and was aware that whether these exposed terminals were live or not was not under their control but they had a disregard for the risks when issuing the work permit. They got fined £100K for this. (The other company got fined nearly three times as much - presumably for leaving exposed live connections that could accidentally be touched and not marking the area as restricted/dangerous.)

      Basically this seems to have been a mega screwup between the management of two companies working on the same site who appear to have had no way to safely coordinate work where the areas of responsibility overlapped.

      --
      God said, "div D = rho, div B = 0, curl E = -@B/@t, curl H = J + @D/@t," and there was light.
  7. Re:Geez, WTF is up with people? by Anonymous Coward · · Score: 0

    What does this story have to do with contractor vs. employee? Idiot.

  8. Shame this had to happen by Anonymous Coward · · Score: 1

    With Load balancing, fail over clusters, hot sites, all the thing that can make part of a site go offline for a while without serious impact.

  9. Re:Geez, WTF is up with people? by Pinky's+Brain · · Score: 2

    He didn't sign a contract which said "you indemnify us even if we appoint a moron to direct your work which allows circuits to be energized completely willy nilly without a care of his workers". We don't allow contracts like that to be signed.

  10. Question by ledow · · Score: 2

    How much downtime was caused by ensuring the circuit was safe and removing the body?

    If zero, SUE THEM INTO OBLIVION. Risking either this contractor unnecessarily (you could have just switched it off) or other workers and emergency workers (because you didn't switch it off after it had demonstrably killed someone).

    If some downtime, then why couldn't you have done that to do the work?

    Sorry, but I fail to see how the risk of a death and possible short-circuits, joined phases etc. because of working with the terminals live in any way "secures" uptime any more than scheduling proper downtime and having properly redundant systems.

    You are just ask likely to bridge the WRONG circuit while working live, or causing a short, which will cause more damage and more downtime than just switching things off to do the work. And you guys have redundant power with UPS that you can bypass to work on the UPS, etc. if necessary? If not, that downtime isn't all that important to you anyway.

    There's no excuse for this, hence the court fine. And you've got to be an idiot to knowingly let people work on a live multi-phase system. Hell, even a fused, RCD'd, single-phase can be bad enough.

  11. From my experience by l0n3s0m3phr34k · · Score: 4, Interesting

    In working in data centers, I can totally see how this happened. Reading the actual source article, it reads like they had already connected the first circuit, and he got popped while working on the second. I would assume they had shut off both, installed the first PSU, then probably someone turned them BOTH back on instead of just the first one. When he went to connect the second PSU...

    These are the kinds of accidents proper "change control" is supposed to stop, it seems no one working there really knew the over-all implementation plan. At our local data center, we have actual licensed electricians for high DC stuff, they know to "never trust always test". Even though we contract all that out too, we try to make sure the people on the site are aware of these things via bright stickers, lock-outs, etc. I have no idea if they have required licensing and training for their "cable jointer" positions in the UK.

    1. Re:From my experience by thegarbz · · Score: 3, Informative

      That's the kind of stop lock-out-tag-out is supposed to prevent.

      I question how it was possible for someone to accidentally re-energize something.

  12. Nanny doesn't know best. Ever. by Anonymous Coward · · Score: 0

    Someone made the call to have live circuits worked on, despite this being illegal.

    But IT SHOULD NOT BE ILLEGAL. If some people are prepared to take the risk they should be allowed to. The magic of the market means they be able to command a higher rate than the do-nothings and gate-keepers who insist on doing it "by the book". Ever had IT refuse to install something because "you didn't raise a ticket" - same thing. Lazy scum the lot of them. They only get away with it because of unions.
    --
    roman_mir

  13. One argument for switching to DC voltage by slacklinejoe · · Score: 1

    Management doesn't understand the difference between telling them it's technically possible to do live maintenance and that it's a challenge like the rest of our technical feats. I feel for the guy and his survivors. I've seen the same pressures play out on my data centers, but thankfully we were able to arrange that type of work as semi-regular full DR test and the place I was working at simply didn't have the same uptime demands when push came to shove. From a safety angle, I can't help but think that moving over the DC voltage for datacenters might be a better option. Some of the big boys are already doing it as we're currently taking AC power, running it through or along side DC power backups, then up to AC for power supplies, then DC for the servers. There's an economy to it, but DC is just safer to work with. That said, there's always going to be an AC/DC bus that's dangerous to work on and live electrical work on it will always take appropriate design ($$) and proper training.

    1. Re:One argument for switching to DC voltage by shortscruffydave · · Score: 1

      DC is just safer to work with

      That depends...if you touch a live DC wire with your palm, your hand will naturally contract and grip the wire. Same thing with an AC wire, and your hand will be repelled. With enough power, both can still hurt like hell, but the latter is the better option.

    2. Re:One argument for switching to DC voltage by elal1862 · · Score: 1

      DC is just safer to work with

      Nope. Not at power densities that are common in data centers; you're merely exchanging one risk (of shock) for another (increased arc flash risks).

    3. Re:One argument for switching to DC voltage by Anonymous Coward · · Score: 0

      Hell no! The voltage detector is the most important safety tool in electrical work, and it doesn't work on DC circuits.

  14. How does that help? by Anonymous Coward · · Score: 0

    Does any of that fine money go to the electrician's family? Nope! It goes to the political class who benefited from his death.

    1. Re:How does that help? by ledow · · Score: 1

      The UK has victim surcharges for most crimes. Those go directly to the victim and/or those who are victim of the same or similar crimes.

      And it's a statutory fine for noncompliance in H&S. Now that it's been established by a court that the companies were negligent, the FAMILY can individually sue on that basis to receive compensation for their particular consequences.

      The fact of law for these exact circumstances has been established by a professional body in court. Now the family don't have to pay lawyers to do that part, they can just sue using their precedents on this incident.

      More likely, the company or its insurers will now hastily settle such claims out of court to avoid additional legal expense. That won't come cheap, but cheaper than fighting the family with lawyers when courts have already ruled you should never have allowed it to happen.

      If anything, this is a case where the government bodies have saved the family lots of unnecessary grief, hassle and expense of their own.

  15. Voltage vs insulation by evanh · · Score: 1

    The DC bus is likely at a lower voltage than typical AC circuits, therefore less able to cause electrocution.

    1. Re:Voltage vs insulation by acoustix · · Score: 1

      The DC bus is likely at a lower voltage than typical AC circuits, therefore less able to cause electrocution.

      It's not the voltage that kills, it's the current. As little as 100mA is fatal if it crosses the heart. And if you want to work with lower voltage for the same job that means the current must be higher.

      --
      "A plan fiendishly clever in its intricacies"- Homer Simpson
    2. Re:Voltage vs insulation by SecurityGuy · · Score: 1

      Ohm's law, though. For the same resistance (a human body), lower voltage == lower current.

    3. Re:Voltage vs insulation by Poingggg · · Score: 1

      The DC bus is likely at a lower voltage than typical AC circuits, therefore less able to cause electrocution.

      It's not the voltage that kills, it's the current. As little as 100mA is fatal if it crosses the heart. And if you want to work with lower voltage for the same job that means the current must be higher.

      The higher current goes to the equipment the powerlines are connected to. But that current will not affect YOU in any way. However, if you happen to touch a low voltage line, the current through your body will be lower (Ohm's law) than when you touch a high voltage line. Hence, low voltage is safer.
      Try taking the poles of a car battery (12V) in your hands and prodding your electrical outlet (110/230V) with two metal rods to feel the difference. Don't do it the other way round, chances are you won't be able to do the second experiment.
      (And where is comes to current: your car battery is able to deliver >100A, while your outlet will be fused for about 16A (in Europe, 230V)).

      --
      What person will donate an airborne act of love?
    4. Re:Voltage vs insulation by Anonymous Coward · · Score: 0

      What a load of crap. Stop repeating what you've heard the grownups say.

  16. Working "Hot" is safe when procedure is followed by Anonymous Coward · · Score: 0

    With the right PPE and proper procedure, it is safe to work on a hot circuit.

    The worker probably took short cuts and violated procedure, resulting in his death. But, as usual, those with deep pockets are chased down and fleeced any time a member of the proletariat suffers from his own stupidity.

  17. Lock out tag out by IT.luddite · · Score: 1

    Industry has already developed safety protocols to address this (LOTO) and three way communications to lower the risk of misunderstandings.

    1. Re:Lock out tag out by aaarrrgggh · · Score: 1

      Not to mention non-contact voltage sensors plus a live voltage check with a voltmeter once that is done. Shutting off the wrong breaker is why you have safety procedures and MOPs in the first place.

    2. Re:Lock out tag out by PPH · · Score: 1

      There might have been a more complex set of circumstances leading up to this. According to TFA, the job involved installing a second source of power to the servers. And one source was kept live to maintain data center operations while the other was under construction. There was also a live test being conducted on the new gear and some mis-communication with regard to the system status.

      Parts of the system were hot at certain times per their plan. But this appears not to have been communicated to everyone involved with the work.

      --
      Have gnu, will travel.
  18. Re:Geez, WTF is up with people? by moeinvt · · Score: 1

    Trolling?

    I tend to doubt that the terms of the job stated that there would be bare wires running through the data center. Nobody hired to work as a maintenance person would reasonably expect that such a hazard existed. The idea that he was completely aware of these risks and took the job anyway is speculative nonsense.

    Furthermore, if you RTFS, this was a FINE imposed on the company, not the result of a lawsuit by the deceased's family. I'm sure that the UK has workplace safety regulations which prohibit exposed high voltage wires, reenforcing the idea that a worker would not anticipate finding one.

  19. Feel bad for the guy by Infiniti2000 · · Score: 1

    I feel bad for him, and I hope his family gets a decent settlement*, but have to ask why didn't he check the live line first? When I work on electrical stuff at home, I always check AND DOUBLECHECK that the breakers are off and that no juice is running. And, that's with voltages that won't necessarily kill me. Working with this level of voltage? Holy crap.

    * As stated by some in the UK, it's not likely to be a big settlement. Too bad in this case.

    1. Re:Feel bad for the guy by Anonymous Coward · · Score: 0

      The article was a bit unclear to me. There is a difference between working AROUND live power, and working WITH live power. I don't kill power to my whole house when installing a new light fixture. I cut power to the circuit, then test to make sure I got the right one. In a commercial setting you lock and tag the breaker to prevent it from accidentally being turned on. The fines are likely justified since the company is responsible for making sure employees are following correct procedures, but the employee must have a safety first attitude as well.

  20. Lockout procedure mandatory in the UK too by Anonymous Coward · · Score: 0

    This is why (at least in the US), standard (and required) practice is to place a lock on the source of energy, and retain the only key to said lock. If multiple workers are all working on the same circuit, each worker has his (or her) own lock on the circuit.

    It's called lockout/tagout, and there are SEVERE fines for removing somebody else's lock (and if somebody gets killed, due to your removing his lock, that would be considered manslaughter)

    It's mandatory in the UK too. Often ignored by independent electricians working by themselves, but hopefully less so by those working for large companies....... UK Health and Safety Executive page on the topic: http://www.hse.gov.uk/safemaintenance/permits.htm

  21. Article versus summary by tomhath · · Score: 2

    As I read the article the accident was caused by a screw up in communications by the contractor doing the work. They tried to blame it on the data center pressuring them, but the judge apparently didn't accept that argument and fined the contractor. It seems submitter is looking for evil where there is really just incompetence.

    1. Re:Article versus summary by ledow · · Score: 1

      When your customer kicks up a fuss for you to "just get it done", it's not licence to just ignore your training, skills and safety procedures. Even if they threaten to cut your business, you still can't just say "Oh, let's forget all that stuff this one time because we might lose future work".

      That's not how the law works, and not how it SHOULD work - the contractor shouldn't just cut corners because they're being pressured by the datacentre. They should say "We do it safely, or not at all". They didn't, hence they were fined.

      It doesn't mean the datacenter are innocent in this - to them, their profit was more important than their contractor's safety (and they still have "a duty of care" to their contractors and workers anyway, written in the law). But the contractor should never have complied out of haste or threat of lost business if it meant a potential for injury or death.

      They're both in the wrong. But the contractor had ultimate responsibility for the decision that caused the death.

      "I don't care how you get it done, just get it done" is not free reign to break the law and allow someone to possibly die. But it's not illegal to say that to a contractor either. It's only ACTING on that without regard for safety of your workers that's illegal, and that's the contractor's fault.

      The datacentre aren't blameless here, but they didn't do anything specifically illegal.

  22. Re:Nanny doesn't know best. Ever. by Anonymous Coward · · Score: 0

    Thanks for the basement dwelling sociopath's opinion.

  23. Re:EEW Permit for hotwork required by Technician · · Score: 1

    Unfortunately, the case appears to be improper Energized Electrical Work (EEW) Google EEW hotwork. There are many cases where EEW is performed. Common applications include linemen replacing cutouts, changing insulators, installing new cutouts for new home construction, etc. EEW hotwork requires special permits and tools and protective clothing.

    Death and resulting fines was due to the failure of following proper proceedures for EEW hotwork. LOTO is preferred over EEW, but there are reasons to do EEW. Only those prpoerly trained and follow the permitting process may do EEW.

    Google EEW Permit for more info. Many pepole are killed as in some codes 600V and under is considered LOW Voltage, not to be confused with Limited Energy Class 2.

    --
    The truth shall set you free!
  24. Re:Geez, WTF is up with people? by Anonymous Coward · · Score: 0

    You knew the job was dangerous when you took it, Fred!

  25. Obvious Cause by Contract+Gypsy · · Score: 0

    Operator/electrician IGNORED NFPA 70E standard and associated requirements. Anyone that has spent a few hours in a 40 calorie suit for Arc flash protection knows what a pain life safety can be. Now you know what safety feels like while having wet feet from sweat! Now as a brit, if his contractor demanded he do work without safety equipment/procedures, he should have walked off the job, filed a complaint with the Jobs office and he'd still get paid + be alive!

    --
    Life is in a state of dynamic equilibrium, it both blows and sucks
  26. I don't understand .. by nickweller · · Score: 1

    I don't understand why you would need to work on a live rain. Seems to me like a very badly designed system ..

    1. Re:I don't understand .. by Contract+Gypsy · · Score: 0

      I presume you mean live main. That is done every single day by hundreds of people, there are countless reasons, the most common being zero downtime is permitted from power loss due to running life saving equipment, a billion dollar experiment, or someone can't risk losing even a single word of information you have place out in the internet! DATA VITAL, Life, not so much.

      --
      Life is in a state of dynamic equilibrium, it both blows and sucks