What To Do After You Fire a Bad Sysadmin Or Developer

Esther Schindler writes "The job of dealing with an under-performing employee doesn't end when the culprit is shown the door. Everyone focuses on security tasks, after you fire the idiot, such as changing passwords, but that's just one part of the To Do list. More important, in the long run, is the cleanup job that needs to be done after you fire the turkey, looking for the hidden messes and security flaws the ex-employee may have left behind. Otherwise, you'll still be cleaning up the problems six months later."

Comments

Re:Blame them!

[TheGratefulNet]

its been my experience that people are generally pretty good, some better than others, but I rarely run into an evil person.

companies, otoh, ...

Reassess Your Hiring Practices

[HellYeahAutomaton]

You hired this employee. Chances are you started off with a relationship of mis-trust:
  - You did a criminal check on the hire
  - You did a drug check.
  - You did a credit check.
  - You did personality test.
  - You used Shockley style brain-teasers to see if they could do things other than what their jobs entail because you don't know how to measure skill, intelligence, or talent.
  - You interviewed in a style of hazing akin to a gang-bang. .. And you still were too stupid to figure out whether or not you had someone who could do the job right.

Sorry, but the tone of the summary makes you look like an asshole, and you deserve whatever you get. This is your wake-up call.

Re:Here be Dragons

[Anonymous Coward]

If it's bad enough you should treat it as a bad virus outbreak and build a completely new system in parallel with the old and move the business information to that system and cut off the damaged system from the net. It's a dirty and tedious job but someone needs to do it.

5 years ago I would not have believed you. That's ludicrous and there is no way a system no matter how touched by an incompetent is so bad it needs to be completely replaced (losing all maturity and buy in etc).

And then it happened. In the form of a manpower resource management tool designed for internal use. Won't go too detailed, but at the worst end of it, after burning the entire budget with the thing only partially finished it was rolled into production. And very soon after it was discovered that for a relatively important part the idiot was actually going in and changing fields in the database himself every day to give the appearance that the thing worked.

The guy who was tasked with fixing it basically came back and said "impossible" .. manager didn't doubt it, thing was scrapped, and a quick and dirty one was built on a shoestring budget to replace it that ended up doing far more than the partially "finished" one that had cost much much more.

Re:idiot?

[Anonymous Coward]

As the same time, I've worked with some people who it took way too long to get rid of.

Ah, haven't we all.
I've been in the unenviable position of having to cover for several 'idiots' higher up the sysadmin foodchain who should have been 'let go', I got so pissed off with the nonsense (e.g. why the fuck was someone who hadn't a clue about Linux managing a whole bunch of Linux servers on paper, when I was doing it on a daily basis as an adjunct to the servers I was looking after) and left myself eventually for pastures new..and left them to deal with it. No doubt I'm now the idiot (à la OPs comment) as far as that lot are concerned (and, no doubt so was my replacement who only stuck it out for about 10 months). .

Whilst I'm at it, here's another true story. One job I had, I set up a Hard disk based backup server, as a backup to our main backup server (a networked tape library) for one of our Linux servers. Everyone was informed, location of server and UPS in one of the comms rooms flagged on network maps (and, it had a big fucking label on the front along the lines of 'Secondary Backup Server - Don't touch'.)
Six months or so after I leave that job, get a phone call, the third hand HDs on the Linux server failed (they were warned in writingthat fitting these disks wasn't a good idea, but hey, that's another story), the tape backups didn't have all the data, so where was the backup server I set up?. I name the comms room, the server name/IP number, there was a couple of minutes silence at the other end of the phone, 'oh, the network manager removed that machine from the network three months ago.'
So again, I'm probably OP's 'idiot' (and the writer of the article pointed to's 'turkey') for that, and probably got the blame for the disks failing.

Ex employees are such wonderfully useful scapegoats to cover up the inadequacies of those still employed.

Finally, OP and the writer of the article pointed to are both idiots and boors.

Been down this path...

[Kelerei]

One of my previous employers, a while back, employed an individual who I will henceforth refer to as the Office Freak From Hell (it had various freaky habits: no personal hygiene, odd behavioural patterns, that kind of thing). I kind of ignored it at first (except to avoid it as much as possible), until it was moved over to my team. It didn't take me long to realise how useless it was -- his code was often delivered late, and was always of a poor quality (example: using strings as every variable type -- really, what the FUCK?). Between my manager and myself, we tried to mentor him, correct him and all of that -- we couldn't fire him straight away as South Africa has really fucking stupid labour laws which makes firing a tedious and difficult process at best (and you'd better not slip up, otherwise the fucktard can successfully sue for damages and the old position back). Meanwhile, I was searching for alternative employment (although mainly because software development in Durban is a dead-end industry, the OFFH was a major contributing factor), received an offer that I couldn't refuse from a company in Cape Town, and put in my resignation. I still had to work a calendar month's notice period though (Americans, things work differently over here!).

That's when things got interesting.

My manager and I started the process of handing over all my projects -- most to the rest of my team, but a few went to the OFFH. It didn't take long for the OFFH to piss off one of my soon to be ex-clients to the extent where top level management got involved, the OFFH was finally pulled into a disciplinary hearing (wasn't fired, but received a final written warning), and I had to step back in and clean out the mess. The next day, the OFFH put in for leave on the Friday coming up, went away... and never came back. It was formally dismissed for absconding shortly afterwards.

That's when we found what was really going on. To summarise:

• - The code that would be pushed through to production was often not the same code checked into the source code repository, and the production code was riddled with security holes, backdoors, and that kind of thing. (Since I used the code in the repos for code review purposes, I never picked this up.) A few months after I'd worked my notice period and left, I heard that they ended up writing new, parallel systems and chucking everything he'd worked on, while doing their best to maintain it until the parallel system was complete. (Side note: I left on friendly terms, and I still keep in contact with those guys.)
• - When we went to try to get source code from his machine (see point above regarding the source repos), we discovered a whole lot of background services constantly maxxing out the CPU. We never found out exactly what they did, but given other discoveries, this pretty much resulted in the network team dropping everything and performing a full security audit of absolutely everything.
• - He would often tag in after hours and during weekends. I remain convinced that he was up to absolutely no good during this time, particularly as I am in possession of an IRC log detailing an intrusion he was involved with on the South African XBox 360 fansite around mid-2009.

So, while we thought we were dealing with mere incompetence, in truth, the OFFH was a malevolent fucktard.

All of us involved has learned our lessons -- personally, I'm far more security conscious, and the folks I worked with are far stricter regarding who they hire, development practices and policies, and that kind of thing. As for the OFFH, it seems to have vanished into thin air...