Slashdot Mirror
Vendors Take Blame For Most Data Center Incidents
dcblogs writes "External forces who work on the customer's data center or supply equipment to it, including manufacturers, vendors, factory representatives, installers, integrators, and other third parties were responsible for 50% to 60% of abnormal incidents reported in a data center, according to Uptime Institute, which has been collecting data since 1994. Over the last three years, Uptime found that 34% of the abnormal incidents in 2009 were attributed to operations staff, followed by 41% in 2010, and 40% last year. Some 5% to 8% of the incidents each year were tied to things like sabotage, outside fires, other tenants in a shared facility. But when an abnormal incident leads to a major outage that causes a data center failure, internal staff gets the majority of blame. 'It's the design, manufacturing, installation processes that leave banana peels behind and the operators who slip and fall on them,' said Hank Seader, managing principal research and education at Uptime."
I'm sure outside forces installing things are disruptive. But then are they the primary forces doing installations in general? And if that's the case, then it would be more appropriate to call them simply installation related issues... and that's both common and to be expected.
Install anything new and teething issues tend to crop up.
I've decided to stop wasting my time responding to AC trolls/sockpuppets... so if you want a response from me... login.
When a company tries to get around minimum wage laws by hiring low-paid monkeys to do their design, manufacturing, and installation, they get exactly what they deserve.
My favorite is getting notifications that all our servers went offline. Now typically, that would be at the network (ISP) level. So come to find out later that the entire facility lost power. Apparently they performed an internally scheduled UPS test without letting us know before hand. Well, they completed the test alright. It was a failure.
In that whole event, we ended up with dirty NTFS volumes that needed to have chkdsk ran and one or two servers with a failed drive in their respective RAID5 arrays. Not happy!
Life is not for the lazy.
over worked, understaffed, added three projects this month and only closed one that was already in the works. It isn't too hard, or that it can't be done, it is also we don't have the time to do it right because we're still cleaning up the mess from the last three projects that were "critical" and were over budget and late. We'd be outsourced, but the cost of hiring outside vendor is about 10x what in house staff costs, and they would charge more for each project added.
Which is why I no longer try to do things on "low budget" and why everything I look at is Enterprise level. Enterprise level allows me to blame the vendor, because THEY are the ones that are selling this shit to the PHB who doesn't know how ridiculously over simplified the vendor makes it sound.
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
Nice try -
The reality is that whenever something goes wrong, the vendors/contractors are almost always blamed regardless of who is at fault. It's standard business practice for the customer to bring in a vendor for just this reason - if something goes wrong, they can point at the vendor. The bigger the vendor name the better this works. If you can bring the manufacture(s) in that's best of all. Who can blame you then?
The 'abnormal' incidents where an internal employee is blamed are probably instances where there was absolutely no way for that employee to escape responsibility (ie the syslog entry shows that user logged in, using a one time password token in his possession so that there's no chance of "the vendor has my username and password bullshit", and entering the command 'reboot').
I'm not saying that vendors, contractors and manufacturers don't make mistakes - they're human and from the manufacturer standpoint there are always bugs that are going to cause problems. I'm just saying that the employee / external aspect should be taken into account and thus these statistics taken with a very large grain of salt.
blindly antisocialist = antisocial
I honestly wonder how many of these incidents blamed on outside vendors are actually the result of something the outside vendor did, and not the result of some manager yelling and screaming loud enough to make the vendor do something to shut him up and not lose business.