Slashdot Mirror


Disgruntled Engineer Hijacks San Francisco's Computer System

ceswiedler writes "A disgruntled software engineer has hijacked San Francisco's new multimillion-dollar municipal computer system. When the Department of Technology tried to fire him, he disabled all administrative passwords other than his own. He was taken into custody but has so far refused to provide the password, and the department has yet to regain admin access on their own. They're worried that he or an associate might be able to destroy hundreds of thousands of sensitive documents, including emails, payroll information, and law enforcement documents."

42 of 1,082 comments (clear)

  1. Backups? by anonieuweling · · Score: 5, Funny

    With backups no data will be lost. Oh, those are encrypted?

    1. Re:Backups? by shbazjinkens · · Score: 5, Insightful

      Or they could just unplug it? Lost productivity is better than lost data here, I'll bet.

    2. Re:Backups? by Brian+Gordon · · Score: 5, Insightful

      I don't understand how it's possible to be locked out of a system that you have direct local access to. You should at least be able to pop in a livecd and edit /etc/password from a livecd. If you need to decrypt stuff might as well start cracking the hash.. they certainly have the computing power to do it o_O

    3. Re:Backups? by uncledrax · · Score: 5, Insightful

      (windows systems too.. I mean it is a muni we're talking about..)

      But yes.. physical access to a device trumps all. It's probably something like they only have -one- guy that knows what he's doing.. and he just went from being fired to Fed-pound-you-Penn

      --
      ----- The internet has given everyone the ability to have their voice heard equally as loud.. even if they shouldn't be
    4. Re:Backups? by Anonymous Coward · · Score: 5, Funny

      I'll put good money on him cracking before this article gets 200 comments.

      We're at 204. Pay up.

    5. Re:Backups? by Harmonious+Botch · · Score: 5, Insightful

      Productivity? By a government agency?

      This is not about productivity, it is about control.

    6. Re:Backups? by Anonymous Coward · · Score: 5, Funny

      Unfortunately, he said he would put "good money" on it and all he has is US Dollars.

    7. Re:Backups? by TheRealMindChild · · Score: 5, Insightful

      I've patched programs stored in a DB without knowing the DB admin password, just by hexediting the DB files.

      Worst. Idea. Ever.

      You should be ashamed of yourself, not proud.

      --

      "When life gives you lemons, don't make lemonade. Make life take the lemons back!" -- Cave Johnson
    8. Re:Backups? by HuguesT · · Score: 5, Insightful

      You are being disingenuous at best. Are your roads in order, is the traffic calm and orderly? Do you have electricity in your home? Are you being raided by armed bandits? what about clean water, can you drink the water coming out of your faucet? What about the mail, is it being delivered?

      Need I go on? You are suggesting local, state and federal government do nothing.

    9. Re:Backups? by nospam007 · · Score: 5, Funny

      >You are being disingenuous at best. Are your roads in order, is the traffic calm and orderly? Do you have electricity in your home? Are you being raided by armed bandits? what about clean water, can you drink the water coming out of your faucet? What about the mail, is it being delivered?

      Are you saying if he gives up the password the potholes will be fixed, the traffic will flow, the mail will be on time and the water from the tap won't stink anymore?

    10. Re:Backups? by darkpixel2k · · Score: 5, Interesting

      You are being disingenuous at best. Are your roads in order, is the traffic calm and orderly?

      Nope. It's always backed up and the roads have lots of bumps and a few potholes.

      Do you have electricity in your home?

      Yes, at outrageous rates to California's energy policies.

      Are you being raided by armed bandits?

      No, but I don't need a police force for that. Just a gun. Except the SF doesn't want you to be able to have a gun.

      what about clean water, can you drink the water coming out of your faucet?

      I can't really comment on the water in SF--but if the city wasn't providing it, I'm sure the people could figure *something* out. And their solution would probably be cheaper.

      What about the mail, is it being delivered?

      FedEx, and UPS both courier mail across town and across the country. You can even pay bike messengers to deliver stuff.

      You know--it's really amazing just how many solutions there are that don't end with "we need the government to do X"

      --
      There's no place like ::1 (I've completed my transition to IPv6)
  2. This is why... by Gallenod · · Score: 5, Insightful

    ...you disable his account *before* you tell him he's fired.

    --

    TLR

    A man no more knows his destiny than a tea leaf knows the history of the East India Company
    1. Re:This is why... by Televiper2000 · · Score: 5, Insightful

      I was just about the say the same thing. You also escort them directly out of the building and let them pick up their personal things a week later.

      --
      New! Device Legs: These legs will help your poor OEM installed product escape any hamfistedness it may encounter. Ava
    2. Re:This is why... by martin-boundary · · Score: 5, Funny

      Nah, they should just reboot the system. That always works, I've seen it countless times in movies.

    3. Re:This is why... by Anonymous Coward · · Score: 5, Interesting

      My employer doesn't fire anyone... they just lay them off, with some amount of severance. That way the person has money and can get EI (Employment Insurance - we're in Canada and like to make unemployment seem nicer than it is), and is less likely to try to sue the company for wrongful dismissal or tell everyone about the shady things the company does.

      The employee is usually taken to one of the front meeting rooms under the pretense of an "important staff meeting". As soon as they leave their desk, someone swoops in and piles everything not owned by the company into a box, and takes it to reception. The employee gets their dismissal meeting from their direct boss with someone from HR present, and then they're taken to reception, given their box of stuff, and told to GTFO.

      Network Operations gets the call to reset the ex-employee's password so they can't get in through the VPN (have to keep their account so someone can answer their email, etc), and work goes on.

      The last thing the ex-employee gets to see on the way out is the hot receptionist. Could be worse.

      Sorry for posting anonymously, but I don't feel like getting laid off if someone from work happens to recognize my username.

    4. Re:This is why... by phatlipmojo · · Score: 5, Interesting

      He's a municipal employee. I don't know about San Francisco, but where I live, state or local government employee means union member, which in turn means he's very difficult to fire, except for the most egregious offenses. He's probably had an extensive disciplinary history to reach this point, which means he had ample time to see it coming and set this all up in advance.

      --

      Nice things are nicer than nasty ones.
    5. Re:This is why... by jason.sweet · · Score: 5, Funny

      Unless they are totally incompetent

      They couldn't event successfully fire the guy.

      -- Firefox isn't as as great as people claim it is.

    6. Re:This is why... by cbreaker · · Score: 5, Interesting

      I'm guessing they are totally incompetent.

      I used to work for the State (a very small state) and some dipshit "Security Director" over at the Department of Administration (all our Internet traffic went through there) decided that he didn't like all this traffic coming from my PC to an IP address that matched a "corporate domain name" (it was my own domain, and I'd login to my own webmail.) Basically this guy was (is) paid $150K a year, and all he does is install appliances and watch logs to try and catch people surfing the wrong web pages (he used to be a cop.)

      He tried to fire me for "running a business from my desk" which of course I wasn't doing..

      Anyways, he sent someone down to my office and they took my PC. Vista x86.

      So they couldn't figure out how to login to the machine. The so-called security expert couldn't even create a boot disk or anything to get access. It's not like it was a crazy machine, it was a Dell Precision machine with a SATA RAID card. All they had to do was download the drivers from Dell and make a BartPE or something.

      They basically told me that if I didn't give them my password I was fired. I absolutely REFUSED. Never do you ever need to have someone give you their password. A so-called security expert should know this.

      So eventually I drove over there, typed in my password for them, and drove back to my office. They didn't find anything, obviously, and I got the machine back completely wiped two weeks later.

      So yes, they are DEFINITELY INCOMPETENT! All IT management in state/government agencies are, and most of the people working for them as well. You move up in the government simply by not being fired and putting in more years than the next guy.

      --
      - It's not the Macs I hate. It's Digg users. -
    7. Re:This is why... by BigDaddyOttawa · · Score: 5, Funny

      Paul, is that you? Could you come to Meeting Room 1 for an important staff meeting. Ignore John standing behind you with that box, he's just collecting them to build a fort.

      --
      Sig? SIG? We don't need no stinkin' sig!!!
    8. Re:This is why... by Zakabog · · Score: 5, Insightful

      So yes, they are DEFINITELY INCOMPETENT! All IT management in state/government agencies are, and most of the people working for them as well.

      The problem isn't true for ALL state/government agencies, the problem is -

      I used to work for the State (a very small state)

      A friend of mine worked for the FDNY in their IT department, they knew what they were doing. It all depends on where you work and the quality of IT staff available for work in the area.

  3. Dennis Nedry? by dunelin · · Score: 5, Funny

    Next thing you know, we'll have some dinosaurs on the Presidio.

  4. Countdown... by geminidomino · · Score: 5, Insightful

    Idiotic new law in 5...4...3...

  5. Re:I had a dream... by gEvil+(beta) · · Score: 5, Insightful

    We all dream about doing this to our ex-employer, but he's the one who's had the balls to do it!

    No, not all of us do. Especially those of us who don't do things that get ourselves fired.

    --
    This guy's the limit!
  6. Read the Article - He wasn't fired. by chipmeister · · Score: 5, Informative

    There was an unsuccessful attempt to fire him. The article also mentions that he was essentially spying on people to learn things being said about him.

    1. Re:Read the Article - He wasn't fired. by Anonymous Coward · · Score: 5, Interesting

      I've seen this sort of problem...it's really deadly. If you have somebody who has the keys to the entire computer system, is fully willing to snoop into people's personal data, and also is willing to really do some nasty things, you're in a bad situation. If you're going to fire him, do it fast and without warning...he absolutely can't know it's coming. With someone like that, you can't even discuss the issue via email with any other colleagues (i.e., he's probably reading your emails quite regularly).

      If he has any time to stew about things, then odds are he'll setup a variety of back-doors or other ways he can royally mess things up. In the situation I've seen, the boss knew the sysadmin was screwing around...though there was no hard proof, the sysadmin also knew that he was essentially caught. But in his position, he basically had the office by the balls. It's a stalemate...unless you're willing to dump the guy and completely sanitize/overhaul anything he's touched on the network. And of course, who knows how much personal data he's copied off-site in the meantime.

      Gotta post as A/C for this one...

    2. Re:Read the Article - He wasn't fired. by thelexx · · Score: 5, Funny

      Well, if they had nothing to hide then they have nothing to worry about right?

      --
      "Gold still represents the ultimate form of payment in the world." - Alan Greenspan, 1999
  7. Got to love damage assessments by Shivetya · · Score: 5, Interesting

    Especially when it makes a crime a Felony. That is one of the four felonies charged to him. The other three are all related to tampering with a computer network.

    While this guy is obviously an idiot for thinking he could blackmail a government entity I am quite pleased the security on the system is sufficient to make it hard to get into when strong security is put into place. In other words, nothing annoys me more than so called secured systems having some means of password decryption, let alone the ones that allow admins to see them plain text.

    what is going to interest me is how many years they will attempt to land on him. Just how offensive to society is this type of crime versus murder or rape. It seems that every new crime invented by the government gets stronger penalties than existing ones; if only to make it appear more valid. After all the penalty wouldn't be so severe if it were not really a crime now would it?

    --
    * Winners compare their achievements to their goals, losers compare theirs to that of others.
    1. Re:Got to love damage assessments by damburger · · Score: 5, Interesting

      he will probably get a sentence more than a rapist but less than a murderer. The state considers screwing with it the highest crime, far more so than the plebs killing each other, but there is a limit to what they can get away with if they want a quiet life.

      --
      If we can put a man on the moon, why can't we shoot people for Apollo-related non-sequiturs?
  8. Job Posting by Anonymous Coward · · Score: 5, Funny

    Large municipal department of technology seeking software engineer for a multimillion-dollar computer system. At least 5 years of previous experience required. Must be able to gain administrative access to a system where the password is not known. Hiring immediately!

  9. Re:Just hack *his* hack by Anonymous Coward · · Score: 5, Insightful

    If you need a recognized code of ethics to tell you that sabotaging your ex-employer's system isn't right, then no code of ethics can help you. Unfortunately this guy screws it up for all of the honest techs who work hard to earn the trust which they need for doing their jobs.

  10. Re:I had a dream... by SatanicPuppy · · Score: 5, Funny

    I've been in a position to do this (I was still rooted from home in three systems, and though they changed the passwords, they didn't kick active sessions) and all I did was change the MOTD to "When firing a user with root access, make sure to abort existing sessions."

    Professionalism is key if you expect to be trusted with access to big sexy systems.

    --
    ad logicam Claiming a proposition is false because it was presented as the conclusion of a fallacious argument.
  11. Unpatch windows by Anonymous Coward · · Score: 5, Funny

    Thats why you run unpatched windows, it will take only 4 minutes to get access.

  12. Re:Apparently they dont have other competent engin by wild_quinine · · Score: 5, Funny

    Number one rule in IT. If i have PHYSICAL access to a system i can get in. Some way, some how.

    Government Agency rule number one: If I have PHYSICAL access to a criminal, I can get information. Some way, some how.

  13. Re:Frankly by damburger · · Score: 5, Insightful

    A reputation, based on people with a serious ideological axe to grind. Blind faith in the market producing magical efficiency gains is contrary to everything I have seen during my professional life, both in the public and private sector. From my perspective, I have never seen one bit of evidence to show there is any truth to it outside the imaginations of Tory politicians.

    Furthermore, people like you who are so besotted with 'market forces' did attempt to introduce them to public services in the UK, and it has been an unmitigated disaster. The inability of internal prices to truly reflect the quality of services has resulted in huge waste, massive bureaucracy and a decline of standards. Now, the ideologues are at it again trying to push for a new round of 'targets' in the NHS. They never learn.

    --
    If we can put a man on the moon, why can't we shoot people for Apollo-related non-sequiturs?
  14. Re:RTFA by alexgieg · · Score: 5, Funny

    What do you recommend they do next time, use a crystal ball or ouija board to predict who's going to pull such a stunt?

    Minority Report for system administration activities? Sweet! ;-)

    --
    Conservatism: (n.) love of the existing evils. Liberalism: (n.) desire to substitute new evils for the existing ones.
  15. I did it too, on a smaller scale by DoctorFrog · · Score: 5, Interesting

    I didn't actually intend to. This was about 15 years ago. I got hired to take care of payroll at a warehouse, which was a completely paper-based process. I suggested that I could transfer the whole operation onto a computer and be more efficient. They said go ahead, but for security be sure to password protect it.

    It ended up taking me only a couple of hours to do what had been an all-day job, and naively I told them this and suggested that there were other areas of operation in the plant I could similarly improve. Instead, the next day they canned me - they wouldn't say why, only "It just isn't working out."

    The day after that I was glumly poking through the classifieds when I got the call

    "Hi, how are you doing?"

    "Well, I'm unemployed. That doesn't help."

    "Ah, yes... well. Say, you know your payroll system? It's password protected."

    "Yes, I know. You asked me to do that." A little bubble of joy started in my chest.

    "Well, could you tell me what the password is?"

    "I could... but I don't work for you any more, do I?" Then I hung up.

    Oh, all the raw data was still available on paper, but I'll bet it took them weeks to straighten it all out completely.

  16. Re:Frankly by damburger · · Score: 5, Insightful

    In the scenario you descibre, the streets would become choked with dirty, unsafe buses and traffic would grind to a halt. This, in fact, happens.

    Like so many market fundamentalists, you just can't see how easily your ideology falls flat on its face in the real world, or you would've seen the flaw in your own argument.

    You are essentially laying all inefficiency at the feet of the 'state' - i.e. any actor that isn't an entrepreneur - and then using that as 'proof' that the entrepreneur is more efficient. This is what people smarter than you refer to as 'circular logic'.

    Perhaps, when you've grown up, experienced the real world a bit and stopped reading Ayn Rands bullshit, you might get a clue.

    --
    If we can put a man on the moon, why can't we shoot people for Apollo-related non-sequiturs?
  17. Folks can see the writing on the wall by scuba_steve_1 · · Score: 5, Interesting

    Firing someone for poor performance (as opposed to firing someone for a single unacceptable action) takes time....and MUCH coordination...at least everywhere that I have worked.

    In a decently managed environment, the employee knows in advance that his management views his/her performance as unacceptable since the manager has discussed it with the employee and laid out a plan for improvement. Even an average employee could see the writing on the wall weeks/months in advance...but this individual was also using his administrative access to monitor related email messages.

    If his group comprised even a moderately-sized MIS group, you could pull his admin responsibilities and transfer him to a role with lesser rights during the period of performance review and monitoring...but this individual was most likely hired to do this very specific job...and there may not have been another position in to which he could transition naturally...even temporarily.

    My question - where are the backup tapes? Pull the tapes from a date prior to his manipulation of the system. Presumably, it should not be that long ago if they were ensuring that at least one other admin had routine access to the system. In such a case, they should have known within 24 hours that he had done something. If, on the other hand, he was a one man show, then I think that they are screwed until he gives up his password...which he will. Mark my word.

  18. Not so easy for sysadmins by phorm · · Score: 5, Informative

    It's just not that easy for a sysadmin, especially a major one. For myself, I've got passwords, SSH-keys, and many other access points everywhere in my company. It's not because I want to screw with them, but because they tend to call me at all sorts of different times and I never know if I'll need secure access to the server.

    So, routing rules from home. Public SSH keys on various border-servers with my USB-drive having the private keys, etc. They're all used for doing my job, and if I'm fired (not sure why I would be though) I'll just move on to the next one without tainting my career and doing something stupid to burn bridges. However, I could see a *bad* sysadmin using these same tools and more to entrench himself so deeply that you'd almost have to rebuild the entire infrastructure from scratch to find all the back-doors.

    If this guy was a real dick (but a clever+smart one), knew it, knew he was going to be canned, and prepared for it... then how are you going to know that your authentication methods, your binaries, or even your kernels haven't been messed with in some way? MD5 sums only go so far when you have hundreds of systems tied together.

  19. Resignation Letter by phorm · · Score: 5, Funny

    Here it is...

    Dear Mr. Baker,

    As an employee of an institution of higher education, I have few very basic expectations. Chief among these is that my direct superiors have an intellect that ranges above the common ground squirrel. After your consistent and annoying harassment of my co-workers and me during our commission of duties, I can only surmise that you are one of the few true genetic wastes of our time.

    Asking me, a network administrator, to explain every nuance of everything I do each time you happen to stroll into my office is not only a waste of time, but also a waste of precious oxygen. I was hired because I know how to network computer systems, and you were apparently hired to provide amusement to your employees, who watch you vainly attempt to understand the concept of "cut and paste" as it is explained to you for the hundredth time.

    You will never understand computers. Something as incredibly simple as binary still gives you too many options. You will also never understand why people hate you, but I am going to try and explain it to you, even though I am sure this will be just as effective as telling you what an IP is. Your shiny new iMac has more personality than you ever will.

    You wander around the building all day, shiftlessly seeking fault in others. You have a sharp dressed, useless look about you that may have worked for your interview, but now that you actually have responsibility, you pawn it off on overworked staff, hoping their talent will cover for your glaring ineptitude. In a world of managerial evolution, you are the blue-green algae that everyone else eats and laughs at. Managers like you are a sad proof of the Dilbert principle.

    Seeing as this situation is unlikely to change without you getting a full frontal lobotomy reversal, I am forced to tender my resignation; however, I have a few parting thoughts:

    When someone calls you in reference to employment, it is illegal for you to give me a bad recommendation as I have consistently performed my duties and even more. The most you can say to hurt me is, "I prefer not to comment." To keep you honest, I will have friends randomly call you over the next couple of years, because I know you would be unable to do it on your own.

    I have all the passwords to every account on the system and I know every password you have used for the last five years. If you decide to get cute, I will publish your "Favorites," which I conveniently saved when you made me "back up" your useless files. I do believe that terms like "Lolita" are not viewed favorably by the university administrations.

    When you borrowed the digital camera to "take pictures of your mother's b-day," you neglected to mention that you were going to take nude pictures of yourself in the mirror. Then, like the techno-moron you are, you forgot to erase them. Suffice it to say, I have never seen such odd acts with a ketchup bottle. I assure you that those photos are being kept in safe places pending your authoring of a glowing letter of recommendation. (And, for once, would you please try to use spellcheck? I hate correcting your mistakes.)

    I expect the letter of recommendation on my desk by 8:00 am tomorrow. One word of this to anybody and all of your twisted little repugnant obsessions will become public knowledge. Never f*ck with your systems administrator, Mr. Baker! They know what you do with all that free time!

    Sincerely

    David Blocker

    Network Administrator

  20. Declare him a terrorist by thc4k · · Score: 5, Funny

    1. declare him a terrorist
    2. torture him
    3. ???? [redacted for national security reasons]
    4. password!

  21. The perfect litmus test by Starglider · · Score: 5, Interesting

    If they were using symmetric cryptography correctly, it could be virtually impossible to recover any of the information without first recovering the password.

    Actually, this is the perfect way to test the strength of symmetric encryption algorithms. For those cryptographers with tin-foil hats (http://www.schneier.com/essay-198.html), seeing how long it will take for various three lettered agencies to recover the data will illuminate a previously dark room containing the question, "How safe is your data really?" It seems to me that this guy is doing the whole cryptography community a favor.