Slashdot Mirror
What To Do After You Fire a Bad Sysadmin Or Developer
Esther Schindler writes "The job of dealing with an under-performing employee doesn't end when the culprit is shown the door. Everyone focuses on security tasks, after you fire the idiot, such as changing passwords, but that's just one part of the To Do list. More important, in the long run, is the cleanup job that needs to be done after you fire the turkey, looking for the hidden messes and security flaws the ex-employee may have left behind. Otherwise, you'll still be cleaning up the problems six months later."
After all, everything wrong with the place is the fault of the last person to leave!
... wait, what?
Real mature there guy... With an attitude like that. You'd better have alot of backup plans in place. It sounds like you are a shit place to work for.
Do us ALL a favor. Name your company. So we can avoid it.
...it's hard to imagine the relationship went sour,
"...after you fire the idiot, such as changing passwords, but that's just one part of the To Do list. More important, in the long run, is the cleanup job that needs to be done after you fire the turkey,.. "
The article points out many obvious pitfalls on letting an underperforming employee go, but very few of these problems are unique to the particular situation of letting an obviously underperforming employee go. Most IT departments are pummeled to death with impossible deadlines and demands and management thinks that the complaints and warnings are just "the way it is with those lazy bastards". Truth is, anyone who's worked with IT knows that you have to test your backups and failover procedures, do security audits, tear down setups that are no longer used and keep documentation and automation up to date. BUT first we have to finish this project that was dreamed up by the top level management with absolutely no understanding of the technical hurdles involved. And it needs to be finished yesterday. If you want things to be neat and tidy, you're pretty much expected to take care of it on your own time.
Time flies when you don't know what you're doing
...you wouldn't be asking this question.
The actions necessary depends on what you mean with "underperforming". If that person didn't do much more than sitting in a corner playing games I would say that there's not much to do, but if it was a person taking shortcuts you need to figure out all traces from that person and remove them one by one. And you can't be sure if that was a skilled person.
If it's bad enough you should treat it as a bad virus outbreak and build a completely new system in parallel with the old and move the business information to that system and cut off the damaged system from the net. It's a dirty and tedious job but someone needs to do it.
This also highlights the need of segmenting the network into different segments, one for sales, another for HR, a third for management and then one or more for the operations so that if one segment is compromised you don't run the risk of having everything exposed. Of course - this goes against the process of using virtualized servers since you can't do physical segmentation on a virtual machine.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
under-performing or metrics may them seem to be under-performing??
Made to do the work of 2-3 people??
Pulling 80 hour weeks that lead to errors and under-performing over time.
Companies are large organizations. Each person in the organizaton may concienciously do their job with good intent but without seeing the bigger picture (not their job) and therefore without knowing the consequences of their actions. The people at the top who, in principle, see the bigger picture, are often so far removed from the details of what is happening that they too do not know what the company is doing, except in respect of the shareholders and overall finanical performance. So, the company runs on policy and no one knows what it is doing. The company can be uber-evil when everyone in it is as nice as can be.
The company is more/other than the sum of its parts.
The real dangers are often not the fired employee themselves(if you aren't stupid about it) but the employees that remain. Most people will not install any insidious backdoors just on their own initiative, but if you fire someone in a way that upsets the remaining employees, i.e. publicly embarass them, screw them out of money they earned etc., then odds are someone else IS going to try to install something to make sure that they don't befall a similar fate.
The submitter comes off as an angry, abusive tool. Maybe he should fire himself for having a hand in hiring an "idiotic turkey" to begin with.
It's likely that the developer wasn't all that bad, but stopped giving a shit after being berated by an abusive asshole for umpteenth time.
I tend to side with the critics here, asking if maybe management (including possibly the person posting the original question) are really the ones to blame?
I've worked in I.T. for something like 25 years now, for companies big and small, though the only times I've held a title of "manager", I was really only tasked with managing outside consultants or developers. I've always preferred being relatively "hands on" with the problem solving and system/network administration tasks at-hand, vs. spending my day in meetings and typing up Excel spreadsheets trying to explain what the "team" was doing.
Bottom line? Sure, there are a LOT of people out there trying to get hired in I.T. as support people or sysadmins who REALLY don't know what they're doing. If more companies would let the people actually DOING those jobs interview these people, they'd be able to weed out far more of the bad seeds before they even started. What I see, time and time again, is some I.T. manager who thinks he's simply "too busy" to interview some potentially really good people who apply for positions, and then he gets in a panic when it comes down the wire and he absolutely can't go without employing another person any longer. He winds up asking H.R. to find him someone good, and of course they don't know squat about I.T. so they pick through the resume submissions based on "standard issue" criteria like the college degree they claim to have, or the number of certifications they list. If he does "second interviews" with these pre-selected people, he may just be trying to pick the best of a bad bunch at that point.
But another problem is with how the I.T. workers are managed. You can have some really top-notch people working for you, yet they're made out to be clueless, inefficient screw-ups because they're actually trying to use their brains to decide which tasks on their plates are REALLY most important to the company. Meanwhile, some upper management character is throwing fits about relatively inconsequential items his ego demands be put "front and center". If you're busy working a difficult problem affecting a whole division of the company and by doing so, you didn't get some new computer issued to somebody first thing in the morning ... guess what usually happens? It's that idiot in I.T. who caused the employee not to have that shiny new PC on their desk on time. Nobody's even aware of the work the I.T. guy was actually in the middle of doing.
And here's the kicker.... You can say all you like about this simply being a "lack of communications" issue. "If management was simply kept informed about what I.T. was doing, everyone would be better off." But so many computer problems are of a "need to fix this yesterday!" level of importance, your good I.T. rank and file employees are going to concentrate on getting that done -- not on getting sidetracked with emailing status updates to key people. Management needs to realize that a certain level of TRUST is required here. You have to say, "I don't really know what Joe Q. has been doing the last few days, but that's ok. I trust Joe Q. because when I make an effort to find out if anyone feels Joe helped them with their issues, I get loads of positive feedback that he did." Micro-managing I.T. is almost never wise....
By using terms such as "culprit", "idiot", and "turkey" you indicate that you are a big part of the problem.
Only gross mismanagement would let you get into such a mess in the first place.
It sounds like he is well rid of you.
It is not that hard to set up a service on a Windows server that provides backdoor services. If you have domain admin rights tunnelling rdp or somesuch through it is trivial. They can use outbound polling of http or dns or even ntp to violate your firewall. You can give the service rights of some other person like the cio for example. Those guys usually demand the keys to the harem. From there you can remote to any server or desktop, do literally anything. These tools are readily available and open source, and every serious enterprise IT pro should have and understand them because often your first job is locking out the last guy.
Help stamp out iliturcy.
Prepare, and execute quickly.
After too many actual shouting conflicts with others, and numerous lies ("even I will have trouble upgrading X11") he had to go. First I arranged for our previous guy, who had gone off to be a consultant while finishing his PhD, to return (at his new rate+housing) for continuity. Then I spent 3 hours with the firee, discussing in detail why he had screwed up in so many ways. I gave him the option of quitting or being fired, he chose the latter for unemployment benefits.
We went to his office, I told his assistant to change all the root passwords, and said clearly that I knew he could screw us anyway. That helped a little, and he was so unaware of his misbehavior that no bombs were left behind. My previous guy was on site the next day.
We eventually hired an excellent professional. He's still doing a great job there through many changes after 20 years, although I left that organization a few years after that hire.
I would also advise, informing your legal team of the decision. You could also hire a security firm (one with a good reputation) to scan your network for security flaws. If you take enough measures to protect your customers data then even if he does have a backdoor it won't come back to hunt you. Additionally consider instead of having a single admin consider having an admin team that watches each others actions, that way you are less likely to have a single admin ruin everything for you.
Easiest answer: Run an audit. That is what I do. I run an audit on all access methods and devices and change the Pwd while I am at it.
The easiest answer, pray.
A bad (as in lazy, surly, abusive) sysadmin who left traps will leave them in places not detectable by an audit.
I have yet to go to a business as a sysadmin where they didn't use default passwords (P@ss1234, now how many businesses use that gem) which are on just about every device or local admin account. The smartest businesses had a different default password for each type of device/account but you end up with password reuse across a pattern of devices and accounts. The thing is, almost no business will go around and change this on every single device/server when someone who knows the password leaves.
I left my last position on less than amicable terms (basically they were setting me up to get sacked by giving me impossible tasks, so I chose to leave). The CEO had no clue, but my boss understood I knew the public IP addresses, domain admin/root passwords and router passwords of our 5 biggest clients off by heart. I could see the fear in his eyes when I left (it was senior managements decision to sack me, they wanted to downsize without having to pay anyone out). Of course I'd never actually do anything harmful to that business (they were doing that well enough on their own) but anyone who employs a sysadmin knows that you need to hire trustworthy people and treat them well or it will turn around to bite you in the arse.
Hiring good people and not pissing them off is pretty much the only defence.
Calling someone a "hater" only means you can not rationally rebut their argument.
I have been in IT for nearly 25 years now and have learned a few things along the way. The first rule is that most employees referring to others as idiots, turkeys, incompetent etc need to look first in their own seat.
It is generally a reaction I expect from a dev or sysadmin covering his own faults by passing blame to others. I find most people just want to do what they where hired to do and do it well and given the proper chance and assistance will do just that.
In the last 5 - 10 years though it is generally a result of understaffing and insane deadlines causing less than desired results.
Got Code?
when the culprit is shown the door.
But the person who hired him still works at the firm... that's the real "culprit".
Seven puppies were harmed during the making of this post.
Wow, that's really an useful and elaborative answer. After reading it, one really has a clue about why hypervisor VLANs won't work.
</sarcasm>
The Tao of math: The numbers you can count are not the real numbers.
I hope you are joking. "Under-performing" doesn't mean "idiot" or "turkey" or imply incompetence or malfeasance as TFS would have us believe. To the contrary. someone capable of doing things requiring the type of audit you suggest would probably not be an under-performing employee.
It must have been something you assimilated. . . .
You forgot about hypervisor exploits.
If you must use hardware separation, you ***MUST*** ***USE*** ***HARDWARE*** ***SEPARATION***.
Contrary to the popular belief, there indeed is no God.
Mod up.
"If it can be accessed, it is vulnerable." -Geezer's First Law of System Security.
Scruting the inscrutable for over 50 years.
Backdoors from the current IT person aren't important?
Ah, once again HR proves itself incapable of hiring a good system administrator / employee and instead either went with the cheapest person available or one with lots of certifications and little experience. I'd fire the HR department as well after showing the bad employee to the door.
To be fair, he was complaining about phyisically segmenting the virtual machines that exist on a single physical machine. Of course, that's fundementally impossible, since these virtual machines share the same computing resoruce. His complaint may be a ridiculous complaint, but nevertheless.
Not so ridiculous I think. There was an article here on Slashdot a couple of days ago about the possibility to spy from one virtual machine onto another one running on the same virtual host by observing the cache line eviction pattern. All VM's share a same cache, and by observing which cache lines gets thrown out (presumably due to the usage by the other VM's), it is possible to infere what goes on in these other VMs.
"Why are you requesting three roles here? I thought you just needed a computer guy".
"Having a team adds flexibility and redundancy, for example, if one gets hit by a bus or goes on vacation, the others can cover."
"How likely is he/she to be hit by a bus? And we'll just not let them go on vacation if that's what it takes."
"I doubt we'll be able to hire someone qualified if we don't allow them vacation time."
"Oh, we'll give them vacation time, we just won't let them take it. Or, if we have to, we'll make them carry their laptop while they're away."
"Then that's not vacation, is it?"
"Quit being such a whiner. Oh, and the salary you asked for? Find someone for 60% of that. Revenues are down."
"Didn't the CEO just get a huge bonus?"
"What does that have to do with anything?"
TL;DR: Companies don't make hiring decisions based on what makes sense, they make them based on how little they can spend.
Never underestimate the power of stupid people in large groups.