Slashdot Mirror
Disgruntled Engineer Hijacks San Francisco's Computer System
ceswiedler writes "A disgruntled software engineer has hijacked San Francisco's new multimillion-dollar municipal computer system. When the Department of Technology tried to fire him, he disabled all administrative passwords other than his own. He was taken into custody but has so far refused to provide the password, and the department has yet to regain admin access on their own. They're worried that he or an associate might be able to destroy hundreds of thousands of sensitive documents, including emails, payroll information, and law enforcement documents."
We all dream about doing this to our ex-employer, but he's the one who's had the balls to do it!
Especially when it makes a crime a Felony. That is one of the four felonies charged to him. The other three are all related to tampering with a computer network.
While this guy is obviously an idiot for thinking he could blackmail a government entity I am quite pleased the security on the system is sufficient to make it hard to get into when strong security is put into place. In other words, nothing annoys me more than so called secured systems having some means of password decryption, let alone the ones that allow admins to see them plain text.
what is going to interest me is how many years they will attempt to land on him. Just how offensive to society is this type of crime versus murder or rape. It seems that every new crime invented by the government gets stronger penalties than existing ones; if only to make it appear more valid. After all the penalty wouldn't be so severe if it were not really a crime now would it?
* Winners compare their achievements to their goals, losers compare theirs to that of others.
Number one rule in IT. If i have PHYSICAL access to a system i can get in. Some way, some how.
There Can Be Only One...
He would have snapped either way, they should be thankful he did not do more damage.
No matter what you do, you can't stop stupidity, madness, hatred and malice. If someone is clever enough or in a position of trust, as inevitably someone has to be, this can happen and you can't always predict it. So the problem is not that a disgruntled employee pulled the plug, but that appropriate checks and balances were not in place. If they were, no individual at all would have been given that sort of power. For a single person to bring down a system is the system's fault.
Do it yourself, because no one else will do it yourself. [beta blockade 10-17 Feb]
fedex it. nothing at workplace is private from employer.
By using the fact that they still have physical access? Resetting his password, or re-enabling other admin accounts is trivial if you can boot the target server with a recovery disk or something along those lines.
The lesson here is that a sufficiently large corporation is indistinguishable from government. --ultranova
Been around since the time of Juvenal's Satires (which would be the third or fourth century AD, I think, unless someone wants to look it up and correct me).
Think for a moment. If you are a senior IT administrator or a senior programmer, unless you're in a very rigorous environment, your actions are most likely not subject to peer review. No-one has time. Right?
How many times do we see the argument "it's open source, anyone can read the code" immediately presented with "but who does"? Now consider that there are millions of people using Linux who potentially could read the code and who are likely working with it because they have a personal passion; but a handful of people who potentially could review your work, but are unlikely to have any deep yearning to do so because, well, they've got their own work to do.
In this kind of situation, you either have to have a mandated peer review regime (time consuming and expensive) or an independent audit (ditto). Both of these are, for reasons of practicality, likely to hit only subsections of what needs to be reviewed.
It's a trust thing. If you can trust your admins. And if you can't...well, who admins the admins?
None of us know all the facts of the situation, but I think it's pretty obvious that this guy was just trying to maintain his livelyhood through a misguided attempt at job security. If we had an IT Union looking out for our careers that gave us some sort of protection against the arbitrary whims of upper-management, then maybe this wouldn't have happened.
As for the idea that the guy might have shared his password with some unscrupulous feind... how many of you, had you actually been given admin access to SAN FRANSISCO would really share that password with anyone? Drastic, misguided, sure... but stupid? Come on, there had to be a reason he got the job in the first place.
"Knock the stones together, guys!"
I do not bow to his guts. There is a fine but definite line between fantasy and reality. This might be YOUR data he uses to play his game.
It is not gutsy to do this. It is childish at best. And no, it doesn't matter if he might be in his right with whatever dispute he has or had. Put him in jail untill he is willing to talk.
Don't fight for your country, if your country does not fight for you.
I didn't actually intend to. This was about 15 years ago. I got hired to take care of payroll at a warehouse, which was a completely paper-based process. I suggested that I could transfer the whole operation onto a computer and be more efficient. They said go ahead, but for security be sure to password protect it.
It ended up taking me only a couple of hours to do what had been an all-day job, and naively I told them this and suggested that there were other areas of operation in the plant I could similarly improve. Instead, the next day they canned me - they wouldn't say why, only "It just isn't working out."
The day after that I was glumly poking through the classifieds when I got the call
"Hi, how are you doing?"
"Well, I'm unemployed. That doesn't help."
"Ah, yes... well. Say, you know your payroll system? It's password protected."
"Yes, I know. You asked me to do that." A little bubble of joy started in my chest.
"Well, could you tell me what the password is?"
"I could... but I don't work for you any more, do I?" Then I hung up.
Oh, all the raw data was still available on paper, but I'll bet it took them weeks to straighten it all out completely.
Seems kind of funny that the article reports the DA is "tightlipped" about his motive. Makes me wonder if he is 'disgruntled' for a reason that would embarrass the agency if it got out.
Also pretty funny that they go into great detail about his salary, which seems kind of low to me for the area or at least average. Sounds like they are trying to make him seem unsympathetic in the public eye.
When information is power, privacy is freedom.
My employer doesn't fire anyone... they just lay them off, with some amount of severance. That way the person has money and can get EI (Employment Insurance - we're in Canada and like to make unemployment seem nicer than it is), and is less likely to try to sue the company for wrongful dismissal or tell everyone about the shady things the company does.
The employee is usually taken to one of the front meeting rooms under the pretense of an "important staff meeting". As soon as they leave their desk, someone swoops in and piles everything not owned by the company into a box, and takes it to reception. The employee gets their dismissal meeting from their direct boss with someone from HR present, and then they're taken to reception, given their box of stuff, and told to GTFO.
Network Operations gets the call to reset the ex-employee's password so they can't get in through the VPN (have to keep their account so someone can answer their email, etc), and work goes on.
The last thing the ex-employee gets to see on the way out is the hot receptionist. Could be worse.
Sorry for posting anonymously, but I don't feel like getting laid off if someone from work happens to recognize my username.
Firing someone for poor performance (as opposed to firing someone for a single unacceptable action) takes time....and MUCH coordination...at least everywhere that I have worked.
In a decently managed environment, the employee knows in advance that his management views his/her performance as unacceptable since the manager has discussed it with the employee and laid out a plan for improvement. Even an average employee could see the writing on the wall weeks/months in advance...but this individual was also using his administrative access to monitor related email messages.
If his group comprised even a moderately-sized MIS group, you could pull his admin responsibilities and transfer him to a role with lesser rights during the period of performance review and monitoring...but this individual was most likely hired to do this very specific job...and there may not have been another position in to which he could transition naturally...even temporarily.
My question - where are the backup tapes? Pull the tapes from a date prior to his manipulation of the system. Presumably, it should not be that long ago if they were ensuring that at least one other admin had routine access to the system. In such a case, they should have known within 24 hours that he had done something. If, on the other hand, he was a one man show, then I think that they are screwed until he gives up his password...which he will. Mark my word.
In this case, it isn't even anything sinister. Basically they get a court order compelling him to give up the password. If he refuses, he's in contempt of court and they'll lock him up until he does. If that's for the rest of his life, well then that's how it goes. He has no grounds at all to challenge such an order so any appeals will get shot down.
Basically they can just keep him in jail until he decides to give up the password. Most likely, this wont' be long at all. Sounds like this guy isn't a hardened criminal, just an asshole with an over inflated sense of self importance. I'm guessing after a few days he'll realise how much this sucks, and his lawyer will explain that he is in fact just going to sit here until he gives it up, and that the ultimate sentence he'll get will only get worse the longer he stonewalls.
I've seen this sort of problem...it's really deadly. If you have somebody who has the keys to the entire computer system, is fully willing to snoop into people's personal data, and also is willing to really do some nasty things, you're in a bad situation. If you're going to fire him, do it fast and without warning...he absolutely can't know it's coming. With someone like that, you can't even discuss the issue via email with any other colleagues (i.e., he's probably reading your emails quite regularly).
If he has any time to stew about things, then odds are he'll setup a variety of back-doors or other ways he can royally mess things up. In the situation I've seen, the boss knew the sysadmin was screwing around...though there was no hard proof, the sysadmin also knew that he was essentially caught. But in his position, he basically had the office by the balls. It's a stalemate...unless you're willing to dump the guy and completely sanitize/overhaul anything he's touched on the network. And of course, who knows how much personal data he's copied off-site in the meantime.
Gotta post as A/C for this one...
He's a municipal employee. I don't know about San Francisco, but where I live, state or local government employee means union member, which in turn means he's very difficult to fire, except for the most egregious offenses. He's probably had an extensive disciplinary history to reach this point, which means he had ample time to see it coming and set this all up in advance.
Nice things are nicer than nasty ones.
Pfft. That's irrelevant if you've got physical access. You'd either pull the drive in question and attach to another operational machine, then change /etc/shadow, or you boot from a LiveCD and do the same.
I'd assume there are other layers of security, though (poss. including encryption), and TFA doesn't say what operating system it runs on.
Hail Eris, full of mischief...
E pluribus sanguinem
I would not be so sure. For it to be theft (in the UK at least) there has to be "an intention to permanently deprive"
Without this it is not theft. This is why someone who takes a car for a joyride is charged with "Taking without the owners consent" and not theft for example.
Therefore if it is not the employers intention to permanently deprive the ex-employee of their possessions then it is not theft, and they are in the clear.
Unless you know fully what he has done, you should not continue using it and assume that everything is working properly and will continue to work properly.
;), and a few numbers changed by a few percent.
Typically corrupted data is worse than destroyed data.
At least when the data is gone, the problem is a lot more obvious.
Imagine if the payrolls have been tampered with (payroll files are mentioned in the article) rather than destroyed. And the law (and other) documents have had the word "not" randomly removed in 0.5% of the occurrences
The only problem is if encryption was used AND he hasn't left an open session somewhere which you can somehow get access to.
;).
If the data is not encrypted it doesn't matter if the SQL DB uses passwords or not. Same for the webserver and other stuff.
I've patched programs stored in a DB without knowing the DB admin password, just by hexediting the DB files. Didn't have to wait for the vendor's developers in the USA to get back to us
As long as you have read access to the unencrypted data you have enough access - even if it means changing the drives and reloading the data.
I used to work at a bank. I was the "cash control teller" which means that I counted every single cash shipment into and out of the bank branch. Sometimes 1/2 million dollars.
You know what? It isn't worth it. It isn't enough to live a good life on. If you get caught, the benefits do not out weight the risks.
The same thing with this sort of hack. The guy screwed himself. He's ruined and will serve time in prison. "Everyone" (with any skills) knows you can get into any system you can physically touch.
What is he going to get for his trouble? Will they pay him off and set him free? HA! no way. The worst that will happen is that they'll employ someone's 12 year old nephew to crack the system. Pay him off with a couple XBox games or a new PS3.
I'm guessing they are totally incompetent.
I used to work for the State (a very small state) and some dipshit "Security Director" over at the Department of Administration (all our Internet traffic went through there) decided that he didn't like all this traffic coming from my PC to an IP address that matched a "corporate domain name" (it was my own domain, and I'd login to my own webmail.) Basically this guy was (is) paid $150K a year, and all he does is install appliances and watch logs to try and catch people surfing the wrong web pages (he used to be a cop.)
He tried to fire me for "running a business from my desk" which of course I wasn't doing..
Anyways, he sent someone down to my office and they took my PC. Vista x86.
So they couldn't figure out how to login to the machine. The so-called security expert couldn't even create a boot disk or anything to get access. It's not like it was a crazy machine, it was a Dell Precision machine with a SATA RAID card. All they had to do was download the drivers from Dell and make a BartPE or something.
They basically told me that if I didn't give them my password I was fired. I absolutely REFUSED. Never do you ever need to have someone give you their password. A so-called security expert should know this.
So eventually I drove over there, typed in my password for them, and drove back to my office. They didn't find anything, obviously, and I got the machine back completely wiped two weeks later.
So yes, they are DEFINITELY INCOMPETENT! All IT management in state/government agencies are, and most of the people working for them as well. You move up in the government simply by not being fired and putting in more years than the next guy.
- It's not the Macs I hate. It's Digg users. -
i know this is /. but straight from TFA, one of his supervisors tried to get the guy canned, and Failed, from there on, he had a couple weeks with his usual permissions, and he set up a program to check what people were reporting about him, as well as set (obviously) a time bomb that would only go off if he didn't have access to reset the time bomb that would make him the only guy with a working password.
I think ironically, that someone working there, Disabled his Password (he reportedly gave one to police) then his time bomb went off leaving the system with NO passwords at all,
and to compound things, they've been using the system 'as-is' because they need it desperately, to do daily jobs. what's going to happen when they find out the whole setup was left password less, the past month of data encrypted and irretrievable, and the only way for admins to work on it is by losing a months worth of data?
and here's the thing, TFA is completely tainted with 'worst case scenarios' they totally assume he gave them wrong passwords (ignoring the fact that it might have been a 'time bomb' leaving the system password less) and also assume that he might have given people on the outside access to the system, with no proof... they also think he has it set so he can destroy data with a cell phone, i mean come on, get real he had like a week or two to plan this from when his supervisor tried to fire him, until they finally fired him..
IMO this guy had a personal disagreement with his manager, and was fired because that guy was working full time trying to find a way to fire someone he disliked.. considering he earned an extra 30k as a trouble shooter and was able to pull off a time bomb, i'm sure he knew what he was doing with technology...
https://www.gnu.org/philosophy/free-sw.html
But yes.. physical access to a device trumps all. It's probably something like they only have -one- guy that knows what he's doing.. and he just went from being fired to Fed-pound-you-Penn
Very likely correct- in which case I say, given the number of KNOWLEDGEABLE people who are out of work right now, the politicians get what they deserve for their stupidity.
This is the reason why you need leaders who know more than the people they are leading. Or at the very least, leaders who know not to kill the golden goose.
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
I will try not to be defensive... this episode took place over 25 years ago so all feelings and facts are not crystal clear any longer.
He received a big promotion into the mainstream MIS division of our company (multi-national) after the successful worldwide implementation of the software. This was a low budget, high visibility project we did and together he and I pulled it off.
Six months later he was escorted to the door with a police escort because of the MIS manager feared his retribution when he was let go.
He was super skilled, very smart and self taught but was a loose cannon at the same time.
I knew he was unstable to a degree... I was simply unsure to what degree and gave him the benefit of the doubt. I'm sure the pressure of the project added to it as well.
When does being a little different from mainstream turn into a disgruntled Engineer hijacking a system?
Sometimes that line is very fine. If the SF employee was handle a little differently by management/HR perhaps the system would not be hijacked nor would jail time be involved.
Making the backups was my way of cooling the situation. If he did tip then nothing was lost except his future valuable service and if he did not tip then nothing was lost either. It was purely insurance against what seemed to be a shaky situation.
What amount was a figment of my imagination?
Probably more than I would like to admit but I was supervising (via team leaders) about 30 individuals at the time and he was the only one I felt this way about.
Thanks for asking that question and giving me the chance to re-examine my feelings, thoughts and reflections.
And in the end, the love you take is equal to the love you make
If they were using symmetric cryptography correctly, it could be virtually impossible to recover any of the information without first recovering the password.
Actually, this is the perfect way to test the strength of symmetric encryption algorithms. For those cryptographers with tin-foil hats (http://www.schneier.com/essay-198.html), seeing how long it will take for various three lettered agencies to recover the data will illuminate a previously dark room containing the question, "How safe is your data really?" It seems to me that this guy is doing the whole cryptography community a favor.
You are being disingenuous at best. Are your roads in order, is the traffic calm and orderly?
Nope. It's always backed up and the roads have lots of bumps and a few potholes.
Do you have electricity in your home?
Yes, at outrageous rates to California's energy policies.
Are you being raided by armed bandits?
No, but I don't need a police force for that. Just a gun. Except the SF doesn't want you to be able to have a gun.
what about clean water, can you drink the water coming out of your faucet?
I can't really comment on the water in SF--but if the city wasn't providing it, I'm sure the people could figure *something* out. And their solution would probably be cheaper.
What about the mail, is it being delivered?
FedEx, and UPS both courier mail across town and across the country. You can even pay bike messengers to deliver stuff.
You know--it's really amazing just how many solutions there are that don't end with "we need the government to do X"
There's no place like
Are your roads in order - no
is the traffic calm and orderly - no
Do you have electricity in your home - yes, but it is provided by a private company, not the government
what about clean water, can you drink the water coming out of your faucet - sometimes. Again, it is provided by a private company, not the government
What about the mail, is it being delivered - sometimes, when I moan enough.