IT Crises vs. Vacation: Sometimes It Isn't Pretty

CWmike writes "It's true that IT systems have become essential to business operations, but the successful functioning of the IT department shouldn't rest on any one person's shoulders. All told, vacations serve as mini tests to prove if a department can function when key players are away. That's the theory, anyway. In reality, IT departments sometimes flunk. The results can either be comical or turn out to be a serious wake-up call to organizations that need a better Plan B. To prime your mental pump before your own vacation, Computerworld compiled anecdotes about good vacations gone bad."

Comments

The real problem

[rsilvergun]

at least in the states is, companies have figured out they can get one person to do the work of two and pocket the other guy's salary. I'm seeing this everywhere in the form of longer wait times for services. It's also really screwing the economy because it means there's 1 less job available, so higher unemployment and less money circulating. We're heading back to the 1800s, when our masters argued that idle hands were the devil's playthings, and the lower class would just spend the time drinking anyway... Kiss your vacations (and your 40 hour work week) goodbye.

Re:The real problem

[Ephemeriis]

at least in the states is, companies have figured out they can get one person to do the work of two and pocket the other guy's salary.

This.

I'm working at a small hospital. Our entire IT department is just three people - one clinical liaison, me, and our manager.

The clinical liaison is awesome at what she does, but she can't build a server or fix a network issue or any of that. Her job is to train the nurses and explain the issues they have in terms we can understand and things like that. She isn't supposed to be technical support.

My manager is certainly skilled... But he's stuck in meetings most of the day, or working on grant proposals, or putting together purchase orders, or whatever. He's rarely available to fix technical issues.

Which means that all of the day-to-day support, and most of the longer-term projects, fall on my shoulders. I've been begging for another technical person for months, and it just isn't happening.

So I'm getting stuck working longer hours... And support is still suffering. It takes me longer to get to the little things, which gives them time to grow into bigger things. And the bigger things are getting fixed as quickly as possible, which means corners get cut. There's not enough time to properly plan/implement/train on new projects, which results in more things going wrong...

The end result is that I'm doing sloppy work, and causing more problems, which means more sloppy work... I can see what is happening, but I can't really do much about it. There's only so many hours I can work before becoming absolutely useless (As the 17-hour day I put in last week showed very clearly. I was completely useless the next day.)

I've got vacation time coming up next week... Which I've had to cut short, due to go-live for a new product... But I've still got a few days off. And, while I'm really looking forward to the break, I'm kind of dreading what I'll face when I come back to work.

Re:The real problem

[JoeRandomHacker]

Here's a thought: stop working longer hours; you are just reinforcing management's bad behavior. At this point you are clearly too important to sack, so no worries there. Do your excellent job at a reasonable pace, and keep a backlog of things you have to do, making it available to interested parties who wonder why things aren't getting done faster. And when new work arrives, let them know that while you would be more than happy to fix their problems right away, there is a pile of other stuff to get through first, so it will have to wait. And most of all, stop worrying about it. It may be that nobody ever wises up and get you some technical help, but at least you'll be less overworked, and maybe a bit happier.

Re:The real problem

[todrules]

It sounds like you are really in a downward spiral here. You need to start looking after yourself more and putting your foot down. It is commendable that you work so hard at your job, but at what cost? See about taking your original, planned vacation. Fill your boss and coworker in on the upcoming deployment, and let them handle it.

Re:The real problem

[innerweb]

As always, the problem is management and management not knowing enough about what they are managing,

Most management has two issues they contend with when it comes to IT. One is they can not see how IT contributes to profit, and therefore see it as nothing more than cost. Another is they do not see how IT can help enhance and manage work flow.

For an IT person to be successful, they need to learn how their management hears things, and learn to talk to them in a way they will hear. Which means to get where you need to be and to get what you need you have to sell it by talking to them at their level. That may not be easy. Sometimes it involves golf games, sometimes fishing, whatever it takes to get to know management to understand what it is they see and hear (we all have filters). Once you learn how to communicate with them, then you have to start to educate them. Once they learn how IT truly can work, they will start to let you have projects that they would not have let you have before. Choose these projects carefully. The ones they can see and feel the success of get you *karma* points. These points become spendable for projects you need that they will not understand the benefit of. Do enough stuff they can see the benefit of, and eventually, they will see the justification of having another person.

Should it be this way? Probably not, but it is, so we in IT have to learn to deal with it. Remember that most management is ignorant about IT. And they want to stay that way . Management typically thinks it has too much on its own plate as it is. They manage things like IT by looking elsewhere and saying, hey see what they are bragging about, why are we not like that? Kind of like all those *investment* bankers who collapsed the economy on bad loans and derivatives. Many said, "Not smart", but their bosses ignored them, saying, "Look others are doing it and profiting , we need that profit as well."

We know that is not real management. But management does not care. If it does not bite them today, it is a good thing today.

That is why it is our job in IT to stealth educate our management. It is our job to know these things. It is also our job to communicate those needs effectively. That is where most in IT fall down. It is very hard to communicate IT effectively. It is even harder to do so when you do not have a grasp on the whole of the company's operations. To be able to explain IT in terms the rest of the company will understand, you have to know their jobs and how IT is used to help them. So, one of the reasons IT management is so extraordinarily tough is that you have to know everything about how the company works to be able to do the job effectively. That means not just running the IT department. It means knowing in full detail how the IT department impacts the company as a whole in every nook and corner. It actually means you wind up knowing more than anyone else in the company about how the company works. IT management is the hardest job in any company.

And that is a natural things when you think about what IT truly is in a company. It is everywhere in a company. The phone system, the desktops, the printers, the servers, the network, the data, the data sharing, the personnel, ... Companies work or don't work because of their Information flow. Information flows because of IT. IT becomes the lifeblood of the company. Wrong numbers in inventory, parts are not made. Wrong field size for an import, data lost. Wrong version of software, job might not get done. Nothing in a company is as pervasive as IT.

We could all go back to pen and paper to track things. We could remove all the digital IT in every company. The job could and would get done (well, most would). But, as what cost? This is probably the thing that an IT person has to understand the most to manage the company (not just IT). This cost reduction from using data systems is where management will understand you. But you have to understand it first. Only then can you demonstrate why hiring another person in IT is profitable.

Re:The real problem

[bigstrat2003]

It's like this. I work to enable me to enjoy the rest of my life, not for its own sake. If my life is really shitty as a result of my job, then I might be better off without it.

This is one of those things that happens because we allow it. If IT employees were willing, across the board, to demand proper respect and consideration from their employer, then there would be no problem. Until we stop cowering in fear of losing our jobs, we're going to be screwed unless we happen to get lucky and have a nice manager.

Re:The real problem

[Jah-Wren Ryel]

I used to think like you do. I believed that it is every man for themselves and that money in the bank was the only way to assure my freedom.

But now that I have a ton of money in the bank and am effectively retired and can do as I please, I am not so sure. I am the richest person I know. Everybody else I know still puts in the 9-to-5 (or often 8 to 7 and sometimes weekends) grind.

I don't think it is feasible to expect everyone, or even a simple majority, to achieve what I have achieved. Maybe its because, looking back, I can see that ~90% of my success is due to nothing more than being in the right place and the right time while only 10% is due to my own fortitude. Few people will ever be lucky enough to find themselves in similarly favourable circumstances.

So while I am still strongly in favour of some the things you wrote, like a competitive market for healthcare. I am not so sure that a go it alone approach is ever going to be successful given the array of forces organised against the modern peon.

Re:The real problem

[DaMattster]

Freedom and self-sufficiency are easier said than done. There is an old adage, "It takes money to make money." In poor economic times (i.e. right now) credit can be difficult to obtain for those with good credit history. You yourself wrote, " It is very difficult, frustration and you often go hungry ....." Try telling that to your son or daughter while you are trying to make it on your own.

Re:The real problem

[Jah-Wren Ryel]

Yeah, I considered including the "and take advantage of the circumstances" part. I actually wrote and then erased "and the balls to exploit the situation" because I felt it would only muddy my point which was that no matter how big your balls, you can't get anywhere without the luck part too. After all, the world is jammed packed full of people with giant balls who are still failures.

Re:The real problem

[EdIII]

Quit. Find another job, or at least start looking right now. Even if you safely move one foot to the other like a stepping stone, DO IT.

I am speaking from experience here. Salaried and was working 20 hours a day for over a year and half. I finally did a re-fi and cashed out the equivalent of 3.5 years worth of my salary (I was not compensated well in the first place).

First two weeks I slept 14 hours a day and woke up after having nightmares where I dreamed about cronjobs, databases, and SQL statements. I would wake up screaming with panic attacks wondering if certain processes were done.

Quit.

What it did was completely shot my adrenal glands. I had nearly killed them. Doctors put me on stuff for a year, and it took about that long for me to get back to normal sleeping patterns and feel better.

It truly is not worth it, life is too fucking short. Don't waste your life on this.

If you have children, a wife, or a girlfriend that IS suffering right now. If you have children you are performing a great sacrifice and being a good father, but you could be a better one being there. I can honestly say that I would have rather been a little more poor, eaten a little less, had less conveniences and video games if I got to see my father more.

The reason why this continues in this environment is that WE let it. An awful lot of IT people are not lazy at all. Some are driven to bad behavior and apathy, but more are driven to keep things alive at all cost. It is our purpose. I took it that seriously. It was like the Path of the Warrior and shit like that. The SQL server WILL not fucking do down BECAUSE I WONT LET IT. I'll figure out a way to work with what I have to make things as redundant as possible.

Quit. Quit. Quit.

Find another job, even if it pays only 80% as much, even if it is a different type of job. Quit.

If you continue down this path, it will be you writing this post to a possibly younger IT guy on some website in the future trying to tell him the same thing. Although, honestly, I would have ignored the advice back then and pushed on like a solider anyways... till I physically could not do it anymore.

Good luck.

Re:The real problem

[Archangel Michael]

First off, it sounds like you don't value your life. Until you value your life, it will be made miserable by your colleagues. You and your boss need to figure out the scope of what is "normal" duties and everything that exceeds those duties comes at a budget cost to other departments. The trick is to get the OTHER guy to say "no".

Them: "We need a new ______ to do _________"

You: "That sounds Great. I'd love to do that, but I'm currently doing _______ (list) for the foreseeable future, which of those things would you like me to stop doing so that I may tackle your project? Alternatively, could you budget an extra $150K for our department so that we can hire someone to do ________ (list) while I start some of these more interesting projects?

I don't ever say "no", I tell them what it will take and ask them if they have budget codes for overtime, and the complete project work schedule. It sounds like you don't do bill backs for things and this is where metered work and a nice Help Desk System comes into play. "Can you put in a work order ticket for me" works wonders, because, as you'll find out, that if it isn't important enough for them to do what is needed, then it isn't enough for you to do it. You let them set their priorities (in writing) and you just do what they want you to do.

In short, let them figure out how to get you more help when it is too busy by making them choose what is important to them, all the while you're saying "sure, we can do that for you".

Everything the article says is true, but ...

[Daniel Dvorkin]

... it won't make any difference, I suspect. There are (at least) three big problems here:

• Techie-macho. A lot of IT people have the idea that if you're not working at least twelve hours a day, six days a week, you're not really working. They don't just put up with sleep deprivation, irregular eating schedules, and the total lack of a social life when it's necessary to get the job done; they actively take pride in it. Vacation isn't even on their radar.
• Management ignorance. IT is seen as a cost sink, so there's a reluctance to hire any more than the bare bones minimum staff to get the job done (and often, not even that.) When the inevitable problems occur as a result of this practice, it's seen as proof of IT's inefficiency and an excuse to cut their budget and resources even more.
• (Often justified) paranoia. "If the boss knows the guy in the next cubicle over can take over for me when needed, I'll be the first one out the door in the next round of layoffs." This encourages "job security through obscurity," the creation of needlessly complex and poorly documented systems that only one person in the entire company can understand. It may not actually work, but people think it does, so they keep doing it.

And no, I don't know what the solution is. Just pointing out that it's a structural issue, and collecting anecdotes about how badly things tend to go wrong doesn't seem to be doing much to motivate anyone to try to fix the underlying problems.

Re:Everything the article says is true, but ...

[Daniel Dvorkin]

So you think overwork, burnout, and constant employee turnover are good things? You're either a sadist who enjoys treating other people like shit, or a masochist who enjoys being treated like shit; either way, don't expect rational people to buy into your sick ideology.

My wife came up with a solution

[mbourgon]

We started going more exotic places. Places that don't have cell signal or internet access. If they want to call a boat to the tune of $5-$10 a minute they're welcome to.

Re:My wife came up with a solution

[element-o.p.]

Why would you welcome a call from work to a boat you are vacationing on?

I've spent roughly one extra day this last week alone on the phone from home trying to fix technical problems at a couple of remote sites. If, as GPP stated, it was going to cost the company $5-$10 per minute to have me work those particular issues, that would be (($5|$10)/minute) x (60 minutes / hour) x 8 hours = $2400 - $4800, not including overtime (if GPP gets overtime; I'm salaried, so it wouldn't apply in my case), to fix the problem. That's somewhere between two weeks and one month's salary for another entry-level to somewhat-experienced IT person.

When it starts costing the company more to call me for help when I'm out of the office than it does to hire more help to take up the slack, they'll finally get the picture, and my life will get better.

Re:My wife came up with a solution

[Machtyn]

The obvious point is obvious. If the company is in such desperate need if his services to spend $300 on a 30 minute phone call, then they must think it is serious enough he should be contacted. In such case, he would welcome the call (and probably be quite surprised by it.)

Re:When I leave.....

[itchythebear]

When I leave ... then I turn off my pager...

So I'm guessing the last time you left was sometime in the 90's?

*ducks*

vacation... or delayed punishment?

[v1]

I know anytime I go on vacation, it causes major headaches for those that try to prevent me from being completely buried by the time I get back

But I'm always buried when I return. Then I have at least a week of torture trying to catch back up. People say someone else is going to get trained and certified to serve as a backup for me, but it never materializes.

I think most companies just have to experience the lesson the hard way, by a bus or a plane ticket. And even then, half of them don't learn anything from the lesson.

They're just too short-sighted. All they see is the cost of investment today, not the return of tomorrow. I find it amazing that business majors, managers, and CEOs don't have that skill. They are blind to the benefits to all involved, and are comforted in the peace they find in keeping their heads in the sand.

Re:Law firm fails because of single disk failure?

[paitre]

I'm not at all surprised by this, actually.

In my experience, there is a significant percentage (IME, most, but others may differ) of businesses that rely on Best Buy for hardware replacements. They see additional hardware lying around as a waste, and will not keep spares handy.

I can EASILY see this happening. EASILY.

Another reason for vacations: crooked employees

[John Jorsett]

I'm personally aware thru my late father, who was an accountant, of two companies that had employees embezzling funds for years. One telltale sign was that they never took vacations, because their replacements would have discovered what they were up to. Businesses should insist that their personnel take time off, just to make malfeasance easier to detect.

Re:Another reason for vacations: crooked employees

[pathological liar]

Investment banks do that already. Guess it's not a magic bullet :P

It's worse the other way

[DNS-and-BIND]

OK, let's say you go on vacation and nothing happens. Next time layoffs come around, you'll be perceived as non-essential.

Vacation? we don't allow that nonsense here!

[green1]

At a previous company I worked for I went away for 3 days (friday-sunday) at a company that only worked mon-fri, so it was essentially just a single day off. I made everyone well aware that I would be out of town and unreachable.

I returned on Monday to find my boss had flown in from out of town and was sitting cross legged on the server room floor with one of our servers in pieces all around him. He informed me that there had been a hardware failure over the weekend and that I should have been there to deal with it. After I finished fixing the original problem, and the problems he had created by trying to fix things, and once everything was up and running again, he asked me for my security pass and escorted me off the premises citing "budget cuts".

Probably much better that I don't work there anymore...

Re:Vacation? we don't allow that nonsense here!

[green1]

Well... hard to say, their official reason for terminating me was due to budget cuts, and they did not fill my position once I was gone, it was termed a lay-off and not a firing. (It was actually somewhat satisfying to hear from a co-worker I ran in to almost a year later that the whole IT infrastructure in the company basically fell apart after that.) There's no way I could prove otherwise, so a wrongful dismissal case would have been an uphill battle to say the least. I suppose it's possible that it was all just a coincidence and that being let go had nothing to do with being out of town when the server broke... but it seemed pretty suspicious, especially after they made me fix it before telling me I no longer had a job.

My boss actually gave me a really good letter of recommendation, but I suspect he had nothing to do with the decision to let me go, I'm pretty sure it came from higher up (the recommendation was on his own letterhead, not the company's as it was "against company policy to write references")

It actually turned out better for me in the long run anyway, I now make about three times what I did there, plus benefits (which I did not have at that company), and I enjoy my new job a lot more than I ever did that one.

Re:There can be only one...

[couchslug]

The flip side to being "one deep" is you are more valuable. I would lean towards hoarding knowledge and being on-call. I don't WANT my employer to be comfortable functioning without me.

Be good at giving verbal instructions and at typing them on the fly for emailing.

Vacation? How about WEEKENDS?

[natoochtoniket]

My current company, has no vacations. You simply tell them when you are not going to be there, and they decide if they want to fire you for the absence.

They also do not have weekends. On the Friday before each customary "3-day" weekend the owner declares an emergency that, somehow, MUST be finished by Tuesday.

No one wants to work there for very long. Turnover is very high. Projects don't get finished, precisely because of the turnover. Other projects do get "finished', but don't work, also because of the turnover

The owner doesn't seem to realize that he is sabotaging his own projects.

Re:There can be only one...

[Flyerman]

You are weak to rely on your own knowledge to keep yourself employed. Be good at managing others in IT and you'll be far more indispensable.

You can, and will be replaced, and are foolish to think otherwise.

The real lesson is...

[seifried]

If you are "indispensable" in the sense that without you the IT services can't be maintained/fixed then the company is f**ked regardless. You may go on vacation, get sick, get hit by a car, have a heart attack or simply get a new job. This is true of any job function, but seems especially true of IT, I suspect in large parts because each IT build-out is pretty much a custom job with all sorts of gotchas, exceptions and internal workarounds to address issues, and the system is rarely documented properly.

I find this especially strange since most companies now rely upon IT to carry out basic functions (telephone, email, workflow, etc.) but fail to treat it as a critical service (single points of failure, especially with respect to personal are more the rule than the exception). Oh well.

Every time

[ledow]

Every time I go on holiday, something weird will happen with a system that's been running perfectly for years. It's guaranteed. And it won't just be because I've been kicking that system back into action all the time and "would get around to fixing it", I would get the really esoteric interconnected problems that suddenly crop up out of nowhere and you're never entirely sure you've solved until months afterwards.

However, my employer knows I'm on the end of a phone if it is indeed an emergency. They have called on me in Italy several times. The trick is to take holidays FAR AWAY from your place of work, and then they can't do anything but cope without you. Flying back to fix a company server? No thanks. Not unless you provide DOUBLE the time I'd taken off in lieu as compensation for ruining my long-planned holiday through poor planning / hiring. If a company can't cope without any single individual, then its hiring policies suck. What would you do if he went under a bus and was *never* coming back?

The worst that's ever happened to me is that I lined up my own brother to go into the company should the emergency they were having not be solved by my instructions. It was, however it would have be after-hours, because he works too, but they would at least have someone there who knew the right switch to press, could be talked through a RAID rebuild, etc. and not have to be led every step of the way and incur only a single day's downtime without making things worse.

Think the downtime wasn't important? It was a school with automated billing, parental contact, phone system, heating controls, registration, medical records, salaries, you name it, not to mention IT lessons and exams. Without registration, etc. the school is legally not allowed to open because they have no records of which children are where, no medical records, etc. Guess what? They coped for the day because they had contingency plans (i.e. cancel all IT lessons and do something else instead, catching up again next week, manual financial control, manual registration, etc.). There are very few companies that *can't* carry on if the IT dies. It might be inconvenient, it might mean harder and more work, but it's rarely impossible unless you're something like an ISP or a datacentre.

If there was nobody else suitable to come and fix the problem? Not hard - hire an IT guy to come in. You do have support contracts for your gear and software, yes? Or you could organise an emergency contractor to visit and fix your problem? It's not hard and the only problem there is finding the right guy (i.e. someone who *can* walk into the middle of a mess and at least get something working enough to last until the "real" IT guy gets back).

If you honestly, genuinely can't cope without an employee - you need him to train an assistant, or even two. It won't be perfect but it's better than nothing. Failing that, you need a large enough team that you can do something on the guy's instructions. Failing that, you need your support contracts which pretty much come as standard with business-level hardware/software. Failing that, you need a contractor at short-notice. If you can't do those four and get to a working system of some fashion within 24 hours, you were always going to be in deep shit whenever anything went wrong anyway. What would you do if the guy left and you had to find a replacement? What if he died? What if he suffered amnesia and forgot all the passwords? What if he was arrested? What if, what if, what if. Or you could just do the normal IT-thing and have backups - lots of them.

Nobody is that invaluable that they have to abandon holidays and drive away from their kids to come back after-hours. Sorry, it's just not true, and if it "is" then that's only the company's fault. It's purely a money saving solution rather than hiring someone else to fix the mess - get the guy who's away on holiday and pay him for a few extra hours - it's cheaper than calling on your support contracts or call-out fees for an emerge

I would fire you for that

[sjbe]

The flip side to being "one deep" is you are more valuable. I would lean towards hoarding knowledge and being on-call. I don't WANT my employer to be comfortable functioning without me.

Business is a team sport and you are definitely NOT being a team player. I have fired people for doing exactly what you are suggesting. It doesn't make you more valuable, it makes you a liability. You are putting the organization at risk for your own gain. If you make everything dependent on you and then you get hit by the proverbial bus, your selfishness has endangered everyone who depends on you. Single point of failure is a bad thing and information hoarding makes you a single point of failure. If the people you work for tolerate that kind of behavior from you, they are extremely foolish.

Re:I would fire you for that

[Kjella]

Single point of failure is a bad thing and information hoarding makes you a single point of failure. If the people you work for tolerate that kind of behavior from you, they are extremely foolish.

Many companies don't spend any more time cross-training than they have to, there's always work to do and being productive always preempts being non-productive. Same with documentation, if I have to scramble to the second most pressing issue on your list I'll do that. They are trying to avoid single point of disaster, nothing more so if a mad scramble by everyone else can lead to a small to moderate degree of failure that's acceptable. Many other places are simply on a "when you have time" basis which make it easy to be an information hoarder, because you're simply always take on enough work to be busy. Or if you're simply dishonest, pretend to be busy.

Personally I've had it too much the other way around, nobody really wanted to or had the skills to do what I do so I didn't hoard it, I was trapped by it. In the end I resigned and started at a different company simply because I felt I was stuck being an expert in one area they'd never let me leave. I would actually think that happens far more often, it wasn't until I resigned they really got busy trying to cover my skills with other employees. Of course I had to learn quite a bit about the existing systems at my new employer too, but I could be a lot more focused on what would be rather than every gritty detail of the old systems they already had experts in.

Re:I would fire you for that

[garyebickford]

Me too. I've fired, and seen fired, people who were too 'effective' at making themselves the single point of failure. Usually they were also irritating, whiny SOBs who had a talent for sabotaging others' work.

Conversely, while I'm naturally an introvert and work best on my own, in my present job I've been very successful at using my own efforts to include others in my projects. It's improved my work, it's improved the stability of part of the IT universe that I'm involved in (many people know at least something about my work), it's improved my position in the company. I'm almost certainly considered more 'necessary' as a result. I suppose this could be considered 'dynamic necessity' as opposed to 'static necessity' where 'static necessity' means, more-or-less, "he's the only one who knows how that POS program works."

  By putting my own work into a Mercurial repository, using dotProject and Trac to manage my own one-man projects and encouraging code reviews and release management of my projects into production IT, the others in this small company were gradually introduced to more-or-less modern project practices, and essentially had to get used to using these tools in order to integrate my work into the production environment. At first only a fraction of that group were interested, but now this has succeeded to the extent that the entire IT staff is now adopting essentially the system that I and the others in my group have been using. In the process, several other people have at least reading knowledge of how my code works, and how it fits into the bigger systems. And that integration process is now documented reasonably well so others can figure it out later.

(One clue - the Trac wiki makes a natural place to make notes while one is figuring out how to build a system, and in the process. Those notes are easy to convert into formal docs later. So those weird little facts and real-world idiosyncracies that affect how the code works are documented both in the code, and in the wiki.)

Re:I would fire you for that

[garyebickford]

I've seen you before. You are basically stuck in a box. (the box of your preconceptions and way of thinking) That box is pretty closely related to the 'union thug' box, not to put too fine a point on it. That box pretty well destroyed Detroit over a period of 40 years. That box will prevent you from ever succeeding beyond a low level, unless you change jobs, get an MBA and go to work for the next group of Wall Street sociopaths (no, not all WS folks are sociopaths, but they're always out there - a separate topic).

When you take a job, you are providing a service in return for pay. This is a standard bargain, just like when you get a haircut. The barber offers a service, and you pay the barber. If the barber gives you an attitude, or doesn't do a good enough job, you will go somewhere else next time. If he/she does a real crap job or pisses you off enough, you won't pay him. But if he does a very good job, has a good attitude and goes above the standard, you might tip him extra.

Similarly, if you do a bit above the standard, look for how to make the company succeed, at most (not all) companies you are likely to be noticed, and sooner or later are more likely to be promoted, given a raise, or in bad times, less likely to be laid off. Sure, some companies don't follow that model - but even then, if you have done the right thing, at least _some_ of the folks in almost any company will be available to give you a good reference when you go somewhere else. So it's just good marketing unless you're a scam artist.

I used to work in the oil exploration business. There are about 30,000 oilfield engineers in the world. It's essentially a small town. If a field engineer made a mistake, that was usually not fatal. But if an engineer screwed somebody over, or failed to carry through on a promise, everybody in the business knew about it soon enough. If that happened enough times, that engineer would eventually be unable to get a job anywhere in the business - or would end up working from some scab outfit working with old crap equipment and cutting corners wherever they could.

Just so you know, the above is not a fantasy. It's exactly how things have worked for me, most places, most times, through a 40 year career. Because I always tried to do the best for the company, even when things didn't work out, I could honestly tell potential employers what I did do, what didn't work out, and why. And I never got negative feedback as a result. Potential employers can smell a rat, and unless they are rats as well, prefer not to hire them.

Re:I would fire you for that

[tophermeyer]

You go in, do the very least you can to not get fired, and go home at night. You don't go out of your way to teach everyone to replace you. You don't "cross-train" so that you can fill two or three roles without a raise. You certainly don't answer calls when you're off duty.

Hooray, you are a disengaged employee. You've got a terribly toxic attitude and you do the bare minimum you can to get by. Not surprisingly, this makes you a poor candidate for investment and advancement. Shocking. Understand that this is often a reinforcing situation, in fact it is an archetypical organizational systems dynamics problem.

I don't know where you have worked or what has happened to you to make you so jaded about employment, but trust me that it doesn't need to be that way. Maybe you had bad managers that missed opportunities to keep you engaged. Maybe you are just a terrible employee. Either way, your view of employee-employer relations is entirely too cynical. Plenty of healthy organizations exist that appropriately reward motivated employees.

Re:I would fire you for that

[garyebickford]

The other reply (below) is correct. What you describe has NEVER happened to me. And, if you read the most successful and influential business books, the entire thrust of all professional business management is the opposite. I suggest "The Fifth Discipline" by Peter Senge. This book is one of the few books that has been on the best seller list TWICE, 10 or 15 years apart - if your beliefs were true, that book wouldn't have sold 10 copies. But having done it myself, I also recognize that running a business is a lot like being a single mom - there's never enough money to do everything that MUST be done, and not enough hours in the day.

So no, I'm not delusional. In fact, there is an aphorism regarding 'electrical engineers' (and by extension computer engineers hard and soft, since this aphorism long predates computers) that EEs have been working themselves out of a job since 1900, but every time they invent things to make themselves obsolete, the increased productivity of the new systems results in higher usage of electrical/electronic/comuting devices and in a requirement for more engineers. I'm not saying this is true of all jobs or all companies, but it's been true enough for long enough.

I'm not arguing that this all applies in all cases. But I have noticed that we tend to experience what we expect, and what we act out - if you think and act like the 'bosses' are assholes, well people notice that, and they will tend to be assholes - to you.

I've gotten in trouble at work before. My thing hasn't been the troubles you've noted. I'm hard to manage, arrogant, too smart for my own good sometimes, and I don't back down. If I think it should be done this way, I'll say so. This caused a rough patch at my present company but it all got worked out - at their initiative because they wanted me to stay. Now I'm more 'essential' in different ways, none of which are related to knowing where the secret software is hidden!

Re:There can be only one...

[Sir_Sri]

I off and on contracted for a place where this was hugely important.

What they'd do is come up with some bullshit reason they were giving a bunch of people 2 or 3 extra weeks of vacation but it must be taken within a short time frame (that quarter or the next two quarters), usually times to align with the summer already planned vacations, and sometimes not entirely bullshit.

Either way, if you were gone for about 3 weeks, and no one really needed you for that time off, your job was going to be axed shortly. Maternity leave? No problem, your job will definitely be here when you get back because we'll try not to fill it at all, and if we don't need it, you're gone as soon as we're legally allowed when you get back.

It's slimy, but it's business.

Fault tolerance is a serious problem. If you only have two people who know a system, both of whom work in the same area, and both get the same infectious disease for a week you have a problem. On the other hand, having 3 or 4 people with redundant skills is a waste of money. I can see the appeal of cloudsourcing to a 3rd party in that regard.

On a personal basis, if you don't have something you, and only you can do until the day you retire you're taking serious risk. That doesn't have to be technical of course, you can be the only one who knows how to deal with the crazy redhead secretary in another department who bothers you all the time, or you could be the only one who knows how stuff in storage is laid out or whatever. It's a tricky balance between 'manpower intensive to replace' and 'crippling the company if you get hit by a bus'.

Common issues

[meerling]

You wouldn't believe how many people I've talked to in a panic because they are having an issue and need to access the server, but the ONLY person with the key or password is unreachable. (On vacation with no contact number, not responding for some reason, or in a couple of cases, recently deceased.)

I know security people will often tell you to limit these things, especially passwords, so that only one person has it and it's not written down. Ignore that. You need to control access, but not so tightly that if one thing goes wrong your company is screwed. Always have a password log, and have it stored in a safe and fireproof location. Same with duplicate keys. It's actually safest if there are 2 backups, and at least one kept at a separate location. (In case of fire, flood, building blowing up, etc.) Obviously keep those secure, like in a safe. Is this 100% security on those things? No, but there's no such thing as 100% security, but it will allow you to keep reasonable security and acceptable ability to respond to emergencies. Both are important, and ignoring one to favor the other will eventually leave you screwed.

And follow the same advice for backups, you need them, they may fail, and they can get destroyed just like everything else. (Easier in a lot of cases.)

Re:Common issues

[Glonoinha]

Print them out.
Put them in an envelope.
Seal the envelope, and sign across where it is sealed.
Let 'whoever' know where they are, and if the envelope is opened you want a damn good reason for it.
If the envelope disappears or is compromised without a good reason, you know to change the passwords.

Re:Common issues

[Thugthrasher]

You wouldn't believe how many people I've talked to in a panic because they are having an issue and need to access the server, but the ONLY person with the key or password is unreachable. (On vacation with no contact number, not responding for some reason, or in a couple of cases, recently deceased.) I know security people will often tell you to limit these things, especially passwords, so that only one person has it and it's not written down. Ignore that. You need to control access, but not so tightly that if one thing goes wrong your company is screwed.

This is something a lot of IT (especially IT security) doesn't always get. We are there to provide service for the company. We are there to make things EASIER for the company. If you get TOO strict with your security, you are going to hurt productivity. Sometimes it's worth it (let's say it is the security related to a database containing customer's credit card numbers/social security numbers/etc), sometimes it's not (server passwords, user password resets, etc.). It's important to make the call. Sometimes the best security practices are just not appropriate.

For now anyway

[rsilvergun]

but if it's one thing our ruling class is good at, it's screwing guys like you. Look at the IT field. They spent 10 years frantically training people for jobs that were being sent overseas (or given to H1-B visas). Hell, during the worst unemployment in 80 years they're arguing for MORE H1-B visas. There's even a law firm that specializing in telling businesses how to offer a job without having to hire an American (they're videos on youtube of them, too lazy to google right now).

No go

[rsilvergun]

you're right, it's happening because we allow it, but it's broader than IT. The only way to stop cowering in fear of losing our jobs is to force our society to put up a permanent and complete safety net. That means unemployment benefits that don't run out, ever, and it means not getting our panties in a bunch if some lazy bloke doesn't want to work at all.

Happened once

[PPH]

They panicked at Boeing when they couldn't get me after hours. So I told them to hire backup. They transfered a guy in from IT who (supposedly) knew Perl (and other stuff). So, on the first day when I was showing him the ropes, I opened up one of the CGI files to show him some of our coding style and conventions. He stared at the file for a few minutes
(with #! /usr/bin/perl staring him right in the face) and asked, "What language is this written in?"

Management had a bad habit of calling everyone down to the shop floor at any hour if any little thing went wrong. I had remote login capability and could generally fix anything from home. One night, I get a call. Panic! All hands to the shop! So I ask, "What's the problem? Maybe I can fix it from here." I'm told I've got an attitude problem. So I get dressed and drive in. An hour and a half later, after reading the error message on the ATE equipment console, sure enough, a 15 minute fix from any terminal in the world.

Often, crisis are engineered to make people look important. Build a system that never dies and nobody notices you. Build one that causes trouble (or hire people who move their lips when they code to maintain it) and you get noticed. And promoted. We had a saying at Boeing: Heads roll uphill.

Re:Happened once

[PPH]

Yeah. But I think management has a problem with the mental image of someone working a problem while sitting at their home PC in ratty boxer shorts. Or worse yet .... naked.

Seriously, when managers don't really have any power they need to create the image of having it. They do so by seeing how many people they can get to jump at their command.

Re:Jesus.

[Nogami_Saeko]

I had a support issue similar to this which required only a few mouse-clicks to solve.

Unfortunately my verbal description of the clicks that were required seemed to be getting lost in translation to the "click-er". Fortunately both of our cell phones had video chat capability, so in the end, he was using his phone's camera to show the screen of the computer, and I was telling him where to click. Problem solved and it saved me an hour of driving in heavy traffic.

One of the first troubleshooting techniques I teach new employees is that if an unexpected error message pops up, or an error that they don't understand, snap a picture (screen shot or cell phone) and send it to me. It has probably made 50% of the "It's not working and an error message popped-up, so I clicked it but it's still not working" issues go away.

It's called CMM and CMMI

[thetoastman]

It's called CMM Level 3.

If a service is business critical, it had best be at least the following:

1. Documented

This means the process is documented well enough so that a reasonably experienced person coming in off the street should be able to muddle through it successfully. This also means that there is budget for documentation.

2. Trained

This means that all people responsible (and their backups) are trained on how to perform these critical functions. This means that there is budget for training.

3. Consistent

All people responsible (and their backups) perform the same task in the same manner. This means that there is budget for performance verification (auditing).

CMM level 3 does not eliminate fire drills or IT crises. It does make them less frequent, more manageable, and of shorter duration.

Downtime is lost productivity. Downtime is lost opportunity. Downtime is lost sales. Downtime means missed deadlines. The cost of these issues can (or should be able to) be determined.

Risk mitigation and cost avoidance are critical components of any business plan. If your business organization does not have these components in place, a disaster will occur.

If your business is a public company, expect institutional investors, major stockholders, and the stock market to punish the business when the disaster occurs. If your business is a private company, you may not recover from a serious infrastructure failure.

IT management needs to be able to articulate the implementation costs and infrastructure risks. Business management needs to be able to articulate the business costs and business risks. Senior or "C" level executives need to match costs and risks so that the appropriate CMM level can be chosen and reached.

This is not rocket science. CMM has been around for about 20 years. CMMI (the successor to CMM) has been around for 9-10 years.

My advice to avoid this

[epseps]

Before I was in charge of hiring for my team, my managers employed a guy who didn't know anything because they did not check his references for some reason. He would always call me while I was on vacation because a simple procedure that he should have known was confusing him. Later our company cell phones were switched from Verizon to AT&T and AT&T had no signal in the Aosta Valley in the Italian Alps.

That is where I would go on vacations.

So if you have AT&T, go to the lovely Aosta Valley but do not cross over into France or else your voicemail will be filled with messages.

There are other causes for this...

[Anonymous Coward]

I'm in the IT department at my company.... I'm pretty sure if the Director of the department and our single Network Admin left simultaneously for any reason, the IT portion of the company would completely collapse.

This isn't the fault of people loading things onto them or what not, it's primarily the fault of an unwillingness to share information about the job. The rest of the department CAN'T start taking over to ease their load and their single point of failure, because we don't get told about things we need to know to do that.

It's infuriating a lot of the time.

Be glad if you've got ANY vacation

[hillbluffer]

The companies I've worked for spent as little as possible on I.T. including personnel. They don't CARE about the future; they want money NOW, and hiring more people costs money. As for digging in youjr heels and refusing; with all the unemployment, they can fire you, and hire a replacement in less than a day, and go on with "business" as usual. Be glad if you've got ANY vacation; the trend now is to make I.T. personnel "independant contractors" and give them NO benefits at all, and NO vacation.

LEarn from our Chinese neighbors

[Dainsanefh]

Your suggestion will work if we allow to BAN the republican party and also BAN Christianity.

Otherwise, only civil war and revocation of the constitution will stop the rich white euro-trash from enslaving the rest of the population.

Re:LEarn from our Chinese neighbors

[Dainsanefh]

Heil Hitler!

Re:LEarn from our Chinese neighbors

[Dainsanefh]

Sieg Heil!

Re:LEarn from our Chinese neighbors

[Dainsanefh]

fsdiuh fsuidghiasudghiasdug

Re:LEarn from our Chinese neighbors

[Dainsanefh]

vxcv zgbsdag aewt 32tr we

Re:LEarn from our Chinese neighbors

[Dainsanefh]

dfgasdf gsdg cxb agewrgaerger

Re:LEarn from our Chinese neighbors

[Dainsanefh]

woo hahaha lfsdoad gasdg

Road Trip

[jacobsm]

I'm going on a 16 day road trip later this month. I'll have my cell phone but I'm not going to be able to VPN into work while driving. Last year I took some time off and received work related phone calls, including requests to join conference calls 7 of my 9 days off.

We're understaffed and there's very limited backup support for the high level technical support staff.

IT is always short staffed worse than nursing.

[CAIMLAS]

Good fucking luck.

Chances are, most functional organizations under 400 hundred people have only one or MAYBE two people who can effectively troubleshoot a bad outage. Sure, they may have an IT staff of 3, 4, 5, 10... but chances are, they're not of the 'sysadmin' type. They're frontline support, most likely, and deal mostly with Windows workstations and servers. For any crucial role, there is no more than one capable person on hand in most IT organizations. The pay masters wouldn't hear of duplicated functionality (that's inefficient!).

After all, if something in IT breaks, the worst management sees that can happen (unlike a dead body from neglegence/overworking your staff) is for there to be a fired employee. They don't see the big picture.

Sure, it's nice to feel needed. It's job security. But it's better to be needed and have someone else who can help pull the weight while you're sitting on a beach with a drink in hand, live a little longer, and have your resume ready to go. Being unemployed for a long period of time isn't half as bad as month after month of high-stress environments where you're pressed with "fix it now under pressure" or "I'm completely burnt out and can't maintain this level of service".

Re:The problem is the question

[Macgrrl]

*Boggle* At 2 years in the business you think you know it all? Vacation time exists for a reason, the same as the 40 hour week. People are more productive if they get a reasonable break every now and again (some studies say a minimum of 2 weeks interrupted at a time is required to reset stress levels to base). Even people doing what they love need a vacation.

Vacation not as bad as quitting

[tompaulco]

We had one IT person who handled keeping about 50 servers up and running, 3 or 4 domains, multiple websites, 30 laptops, 3 different FTP servers, communications with the outside agencies that were sending us data, RFPs, customer support, and who knows what else. He quit for another company where he will probably have 1/4 of the job duties, will not have to work weekends, and will get paid more. Our company has not hired a replacement yet. The first IT related issue that came up was a failed backup that occurred about 12 hours after he quit (failed on a weekend). The next one occurred about 24 hours later when a client locked their FTP account because they can't remember their password, and nobody could figure out how to reset it. They eventually had to call him and ask. That debacle resulted in about 200 hours of unpaid overtime.
Supposedly, they are interviewing replacements, but so far I think they are patting themselves on the back for saving money (about half an IT person's salary, as a guess) and spreading around his work mostly to the overworked development group, including myself, who are now getting surprisingly little development done.

perception vs. reality

[Tom]

Giving up vacation days because you couldn't use them, interrupting vacations or cutting them short for work - if you find yourself anywhere near that list, you're a fucking idiot, and the one you're fucking is yourself. And that was the polite way of putting it.

A few years ago, I've had to become a bit of an expert in vacations for business reasons (negotiations regarding vacation times, rules, company procedures, etc.). Two things are absolutely frightening when you do that.
One is that we need vacations at all - for thousands of years, there was no such thing. That's because work has become condensed to a point where it's detrimental to health at good times.
Two is how little almost everyone, employers, employees, even most union people, realize how important vacations and other free times are. I've seen many people crash and burn in those years and lack of vacations, interruptions of off-work time and not being able to "shut down" when you leave work were almost always present and at least contributing factors.

In all those years, I have encountered one group of people who can do that, who can go on without vacations and free days and suffer no ill consequences. These people share two important characteristics: One is that their time is self-determined to a large degree. In other words: They could close up shop and go away for a few days at any time if only they wanted. They have no boss pressure and no customer pressure that would stop them, because they've organized their work so that if they ever need to, they can. Obviously, most of them are self-employed, but not all. A great example is a cobbler who has his shop down the street from where I live: The official opening times of his shop, as posted in the window, are: "When I'm here."
Two, these people work their dream. They do what they want to do, they have meaning in their jobs, and they've cut out as much of the crap as possible, and some that other people thought would not be possible to cut. They never ask themselves "what the fuck am I doing here?".

None of them make a killing. But they make a living. And I try to be one of them over being rich, but hollow. I've worked with too many so-called "successful" people and seen their dull eyes. Five minutes with someone from the group above and you can never go back into the machinery.

Repeat to fade

[Tomsk70]

1) Company wants to maximise profits (painted over as 'saving money') = company follows staff-cutting recommendations that look great on paper.

2) Company then realises that saving money on paper has left them paying more in contractors to make up for the positions they got rid of = company will be happy to do this if it means they don't have to admit that step 1 was short-sighted.

3) Company realises that someone will be shot when the figures are presented in the correct way (e.g. how much staffing now costs compared to pre-step 1) = company gets rid of most of the contractors taken on in step 2.

4) Company now relies on single permanent staff member who eventually quits = company then decides that 'outsourcing is the way to go' for the same reason as listed in step 2.

I've seen this happen over and over and over, at banks, hospitals, investement companies and councils. In every single instance, they weren't employing enough people to begin with before even reaching Step 1. But the one thing that remains constant is that regardless of the outcome, companies will do *anything* rather than admit a failure and take a step backwards.

At the bank (one of the biggest), for instance - before the crash;

"This is Bob, make sure you all spend time with him, we're taking on another 150 servers from X branch"

"This is the third time this has happened"

"That's correct. Any questions?"

"What are you going to do when things go bad?"

"What?"

"You've sacked another team"

"Yes"

"You're giving the full-time work they were doing to another team that already has full-time work"

"Yes"

"Your share price that you force us to look at every time we open a browser has gone from £3.50 to £7 in less than a year"

"So?"

"So this isn't belt-tightening, this is greed"

"What's your point?"

"My point is - you're acting like companies do when they're against the wall or about to go down, even though this company isn't. So considering you're still behaving as if it is, I'd like to know what you're going to do when things really *do* get sticky, as opposed to pretending they are right now in order to maximise profits".

"......moving on to the next item on the agenda...."

Re:There can be only one...

[BrokenHalo]

First thing I tell people who work for me is...

If you are the sort of boss who feels the need to come out with a statement like that, then you are quite possibly the sort of toxic boss that inspires this sort of self-centred behaviour. Relationships are a two-way process. Many worthwhile employees tend to enter a job with all sorts of useful ideas, only to find the corners knocked off them very quickly as the realisation sets in that the (mis)management issues in the company have a long and repetitive history.

It really does not take very long to take the shine off an employee's goodwill. You don't have to be a doormat, but it doesn't cost you anything to allow people to feel that their input has some value.

Can't be replaced = Can't be promoted

[sjbe]

In the end I resigned and started at a different company simply because I felt I was stuck being an expert in one area they'd never let me leave.

If you can't be replaced, you can't be promoted. Truer words were never spoken.