Slashdot Mirror
Security Plans for When Your Senior Developer Leaves?
An anonymous reader asks: "Our CTO, responsible for all hardware and networking setup, who also coincidentally happens to be our senior (and only) developer, has just resigned to go work for the competition. We are not a software company, but he's written proprietary code that we use on a daily basis to work. What interim measures should we be taking to ensure a smooth transition to the next person hired to take over? What can we do about security, since this person designed and implemented all current security procedures? What about ensuring that we have all the intellectual property to which we're entitled? As one co-worker put it: 'His resignation was a surprise to us, but it definitely wasn't a surprise to him.' If he wanted to leave some hard-to-find malicious timed-release back-door-opening code running, it's certainly within his means. We don't expect any malicious action, and can rely on a reasonable level of co-operation and documentation before he goes, but I want to get a sense of what others have done in this situation."
Get your lawyers lubed up and ready to go.
There are no trails. There are no trees out here.
You should phrase the question "what should we do the next time"
This is a legal problem. I assume that you have all of the agreements in place (signed NDAs, Non-Competes, etc.). So from there just monitor what is going on. Frankly if you are loosing your CTO, only developer, etc.you are screwed as a company anyway, so maybe it is time to update your resume and get a head start on the new job search that is most likely coming your way
I have mod points and I am not afraid to use them
When you hire the person, make them sign something saying that any proprietary code they develop for the company becomes the property of the company. also, insert clauses with penalties for intentional security breaches, etc. it's all a matter of planning. when you hire someone, you want to bring them on, but you should also look at it from the perspective that they can do real damage to your company. you should have them sign something to the effect that they shouldn't do that damage, and if they do, they will be held responsible for any intentional damage. NDA's while not always enforceable if they are unreasonable are a good deterrent as well.
I'd stick to gouging out his eyes and cutting out his tounge, lest he bring a plauge upon your house.
Or, you know, change passwords, and stuff. I hear that works too.
What were you expecting?
In the heyday of the bubble, jumping ships was a practice that everyone knew about it, and often tried. So, relating from my experience of that...
I think your requirements for a replacement CTO should start with securing the system. Hire consultants until the right guy is found to document what's going on - NOT for more development.
Although I have no info about the politics, your lack of insight into managing your technology is stunningly poor. I hope you pick the best of those consultants and hire them to spread this risk in the future.
Above all this, prepare for your competition to now exploit any weakness you have in your market. No more BusinessAsUsual. If you didn't care about what this guy until he left, perhaps you should re-evaluate what you use your technology for, and take it a bit more seriously.
mug
but I want to get a sense of what others have done in this situation."
Ask him if you could go with him to the new corporation.
"Provided by the management for your protection."
You say you're not a software company, so obviously the code that he's written isn't your product. Is it utilities, or something that manages your workflow and process? If so, it doesn't seem like you've got that much of a problem. I guess a lot of it depends on what terms he left as -- going to the competition doesn't necessarily imply a bad breakup, but the tone of your posts seems to. Well, anyways...
The easiest source of information is going to be him, himself. It doesn't sound like he's left on the worst terms, and, really, the truth of the matter is he's got all the cards now. If he wanted to screw you over with a malicious time bomb, he could, and there's very little you could do about it. So I would just take what he gives you in terms of documentation and all, and, unless evidence proves otherwise, assume that he's on the up and up. You have little choice, and the other options (like lawyers) are going to make him very uncooperative. Most programmers I know don't get malicious unless they feel that they've been royally screwed over. YMMY.
But, to the future! The best way to avoid exactly this kind of thing is to not have a new guy, but two (or more) new guys. Even if its a senior-level and a junior-level, having someone who can be your backup is invaluable. At worst (depending upon the software), you could get an intern or other low-paid peon to serve as the backup on the cheap. Some of them are clods, but some can be quite smart. Code review reduces not only bugs, but logic bombs and backdoors, and it leaves someone who at least has a clue about the system if one of the two leaves.
As for security: Make sure you have a firewall, and the rules are set to the bare minimum allowed in (but you should have this already, right?) Change the root/administrator passwords. If you have a competent sysadmin, have him monitor for unusual activity...but these are all things that should be going on all the time. In other words, nothing out of the norm.
If you think they are the type of person who may do something like that, you probably shouldn't have put them in charge of security.
We don't expect any malicious action
Well then you shouldn't have made life so difficult for your CTO. I mean, everyone has their price, PAY IT!
Oh wait, you want team players. Well then who's idea was it to cut his pay, deny funding to the latest project, or take photos at his last "business trip". Certainly not his...
Antiquated competence won't be a job skill forever.
First, hire a security team to secure your systems.
Make sure they remove all existing accounts on all systems, and start with new ones, with very secure passwords. This is a good time to require a password rotation policy, and password length & strenght requirements as well.
No non-secure connections to non-public systems from outside the company, period. Or at all, if you can get away with it. No connections from dynamic-IP connections to internal systems, either. (make sure all allowable connections to internal systems are from a list of known IPs)
Make sure PHYSICAL access is secured! Lots of ex-employees keep security cards, keys, etc, and can often get back in after the fact.
Make sure your people know about 'social engineering'!
Don't use inherently-insecure technology from companies who don't give a rat's ass about your security. No bonus points for correctly guessing which company I'm talking about. This becomes stupendously more important if you're the sort of silly-ass company that only has one techie on staff at a time. Lots of updates are to be applied, no matter what platform you go with.
Now's the time to separate systems if you host stuff. Hosting stuff should go in a co-lo facility (since you obviously don't have the staffing resources to handle your own data center), and you should have separate systems for business needs, like e-mail, etc., in case your website gets DOS'd, it won't impact your e-mail, etc.
Have regular security reviews by external security companies. Rotate which company you use each time.
Make sure your insurance covers all your computing infrastructure and eventualities (fire, flood, theft, cracking, etc.).
Make regular backups.
TEST your backups.
Make sure you have off-site backups.
Make sure you have a disaster preparedness plan and the appropriate people know how to implement it. What happens to your business if the building burns down? If the phones go out? If the Net connection goes down? What if there's a major terrorist attack in your city and noone can get to work? Welcome to the real world.
Make sure you have onsite spare parts for your computers, at least for the critical ones.
Make sure noone saves important documents ONLY on their own machine - either make them start saving to shared drives which get backed up daily, or have each machine backed up daily. Say you lose the business plan you're showing to investors tomorrow? What do you do? WHAT DO YOU DO?!
Don't get locked into proprietary file formats, or you may never be able to switch. Plus you may get hit with 'requests' (ie threats) to inventory every piece of software on your site.
Definitely have more than one techie & programmer (2 of each, at least) at your company. That's flat-out ridiculous, as you are probably aware by now.
Okay, that's all I can think of off the top of my head right now. Have a day.
Get your lawyers lubed up and ready to go.
I think what you meant to say was "Get lubed up and get your lawyers ready to go." They are lawyers, after all. >.>
This statement is false.
We make DSL equipment. Shortly after a layoff last year, all of our test stations at several contract manufacturers stopped working almost simultaneously. It seems one of our test engineers had programmed them to phone home to his PC at headquarters to make sure everything was ok. Thank goodness it wasn't one of the linecard software guys or we could have had thousands of lines out of service.
Trouble? Yes, we've had our Phil.
For the most part, if they were really malicious, you are boned anyhow.... The good news is development is really a small community - even if they don't get the book thrown at them, I know folks that were more or less excommunicated because of bridge burning and other stupid departure tricks. More than ever, jobs are had by personal recommendation rather than some recruiter pushing your resume. You may not like your job, your peers, etc - but I've seen prospects burned before they got in the door because of what they did a company or three back. Odds are, if this guy was a senior level developer, he has more at stake than you. I know I made sure everything was checked in, documented where possible, and asked IT to change my passwords - I also never checked to see if they did...
+++ UGUCAUCGUAUUUCU
Then, once he's gone, audit the code. Maybe you'll need to hire an outside consultant to do it. Anyway, once the source code is audited, you still aren't in the clear. It could be that he put a backdoor in the binaries, leaving the backdoor out of the copy of the source code he pointed you toward. Thus, once you are done auditing the code, compile it. Do a file compare of the current binaries and the newly compiled binaries.
In Windows, the command line is fc /b filename1 filename2.
If there are any differences, that doesn't necessarily mean anything significant. Move the current binaries to a temp directory or someplace out of the way. Don't delete them, as they could be important later. Copy the newly compiled binaries in. Test the whole system to make sure it works.
As for ensuring your intellectual property is protected, I don't know how you can truly do that from a technical standpoint. You should notify your corporate legal counsel of your concern. If you don't know who that is, bring it to your CEO's attention.
Good luck.
first, treat the person leaving with respect. if this person is mature then they won't burn bridges - neither should you.
don't accuse him of things he might not have done. don't screw around with his career. shake hands, wish him well and generally be professional. it's business. cope.
second, solve your problems. the person who is leaving has his own issues - poor communication, poor loyalty, excessive greed, whatever. those are his problems. let him work on those, they're not your problem.
the main reason for your discomfort is that you put all your eggs in one basket. and now your basket has left. so in the future hire two people, not just one.
and when you have these two people on board, talk to them more often. find out how they feel. you were taken by surprise by this person leaving, that suggests poor communication - on his part or your part.
finally, you seem to have no idea what code this person wrote even though your business seems to depend on it. does the code go in a source code control system? do you have a release procedure? can you get the previous releases?
you need to answer yes to all three of those. if you don't answer yes to all of those now, make sure you can in the future.
US Citizen living abroad? Register to vote!
Quite honestly, your company needs to get their ducks in a row. Here is what you are up against :
...
Your company sounds small enough that they had very few 'computer guys' but big enough that the computer infrastructure is fairly complex.
The guy in charge (your soon to be ex-CTO) probably designed and built the existing systems from the ground up. As he didn't have anybody watching over his back, do not be surprised if there is some jury-rigging in there. He probably shared some of the quirks with some of the other computer guys, but not all.
He may be an important part of the wet-ware in your system. An easy to understand example would be a bowling alley - if your company has to bowl a strike every time the ball gets thrown, he was the guy that walked down the alley continually making minor adjustments to the path of the ball. This could be custom reporting on your data, swapping out the backup tapes, deleting temporary files, cleaning out the log files so they don't fill the hard drive, or booting the servers in a particular order so as not to overload the UPS. It has become routine that he takes for granted and probably doesn't even think about them any more so when he doesn't mention it (and they don't get done)
You have some pretty important apps that he may be the only guy that understands how they work.
Today is the day of truth, you are on the cusp of finding out if he is disgruntled or not. If he is disgruntled, the LAST people you want talking to him is HR. They will either piss him off more than he is, or try to bully him - you need to get his favorite tech to take him off-site, dinner or to a strip club, and off the record find out why he is leaving, what his primary concerns are, what he would honestly have changed given the chance, what he anticipates the hot-spots being after he is gone, and most importantly : does he have any recommendations for a good replacement.
This last one is key. There are lots of paper tigers out there (MCS* certs), lots of guys that are good at network administration, lots of guys that can code language A or B or C++, lots of guys that can diagnose and maintain an SQL Server, and lots of guys that can operate in the role of CTO to work as manager and liason between the IT department and upper manglement. You are going to find precious few people that can do ALL of the above(*), and unfortunately that is exactly what you need to do - and find a guy that enjoys doing it because the first few months are going to be rough. Doubly rough if your CTO is disgruntled, so if one of trusted colleages was in there he might hesitate before setting off some time bombs that his pal is going to have to clean up.
The penalty for getting this wrong is going to be pretty severe.
(*) I would be perfect for the job, but I am pretty happy where I am.
Glonoinha the MebiByte Slayer
This is -exactly- what I was talking about when I said the last people you want dealing with this guy is the HR department.
Might as well steal the toys off his desk, decline the steak dinner on his expense report, deduct some personal long distance calls from his paycheck, and key his car while you are at it.
Glonoinha the MebiByte Slayer