Slashdot Mirror


San Fran Hunts For Mystery Device On City Network

alphadogg writes "With costs related to a rogue network administrator's hijacking of the city's network now estimated at $1 million, city officials say they are searching for a mysterious networking device hidden somewhere on the network. The device, referred to as a 'terminal server' in court documents, appears to be a router that was installed to provide remote access to the city's Fiber WAN network, which connects municipal computer and telecommunication systems throughout the city. City officials haven't been able to log in to the device, however, because they do not have the username and password. In fact, the city's Department of Telecommunications and Information Services isn't even certain where the device is located, court filings state."

29 of 821 comments (clear)

  1. The story keeps changing. by khasim · · Score: 5, Insightful

    From what I've read, his "hijacking" was limited to refusing to give the passwords to his boss whom he considered an idiot.

    Given that they cannot hunt down a single device on the network, I'd have to agree with that assessment.

    MAC address ... switch port ... it should be easy.

    1. Re:The story keeps changing. by DogDude · · Score: 4, Insightful

      1. Your boss is your boss. Unless there's the chance that somebody could be physically hurt, your employer's passwords are NOT yours, no matter how stupid you think your boss is.

      2. Assuming that they have wireless on their network, there's no way to find wireless devices, since they can be put inside of locked buildings. Unless your name is "Superman", there's no real way to find exactly where wireless devices are, as far as I know.

      --
      I don't respond to AC's.
    2. Re:The story keeps changing. by goose-incarnated · · Score: 4, Insightful

      ... Unless your name is "Superman", there's no real way to find exactly where wireless devices are, as far as I know.

      And exactly how would superman find it? Xray vision? How would he then know he found it?

      --
      I'm a minority race. Save your vitriol for white people.
    3. Re:The story keeps changing. by moderatorrater · · Score: 4, Insightful

      Agreed. If they're still having problems at this point, they're incompetent jackasses. However, that's not an excuse for the employee to be a jackass too.

    4. Re:The story keeps changing. by Crudely_Indecent · · Score: 5, Insightful

      If Superman had any IT skills, he'd perform a traceroute to determine the devices gateway. Once the gateway was determined, block the mac address from accessing the network. If the admin of that device is worth his salt, he'll change the mac address and continue. They could then specifically enable allowed devices and forbid all others.

      Forget finding it, make the network inaccessible.

      City of SF Admins, if this proves to be your resolution, you owe me $150 for 1 hour of my time. Sorry, I do not bill in lower increments.

      --


      "Lame" - Galaxar
  2. Siding with the network guy by John+Jamieson · · Score: 5, Insightful

    Man, the more I read about this story, the more inclined I am to believe the network admin.

    He may be incredibly bull-headed and lacking social self preservation techniques, but he may have been technically right.

  3. not necessarily wrong... by damn_registrars · · Score: 5, Insightful

    your employer's passwords are NOT yours, no matter how stupid you think your boss is.

    Refusing to give out passwords to higher-ups is not always the wrong thing to do. If you are the network admin, and your job is to maintain security of the network, wouldn't it be reasonable to refuse to hand out passwords to people outside of the network administration roles?

    Although I can say that an admin can make that choice at his or her own peril. After all, the higher-ups can always opt to fire the admin and replace him or her with someone who is willing to seek security of their job over security of the network they are paid to administer.

    --
    Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
    1. Re:not necessarily wrong... by Lonewolf666 · · Score: 4, Insightful

      Agreed.

      If a boss I don't entirely trust demanded my password, I'd offer to upgrade his account to the same privileges at mine, but he'd NOT get MY password.

      The reason is that if he does something stupid that will show up in logfiles, he can damn well do it on his account and get logged doing so ;-)

      --
      C - the footgun of programming languages
  4. Re:to quote bash.org... by FireStormZ · · Score: 4, Insightful

    The admin might not be stupid he might be an ass

    1) He placed a rouge device (his personal property) on the SF network
    2) He set all the network devices on the network to lose all info on a reboot
    3) He will hand over the passwords (after jail) to all the devices except the rogue

    You can make equipment hard to find ( mac masquerading comes to mind )... I'm only adequate in terms of networking but I am pretty sure someone who is really good can play a mean game of hide and seek. Who knows *what* he was doing with that device? and were I the network admin I would have to *on principle alone* rebuild everything after this guy left..

    --
    "Ahh! Arrogance and stupidity in the same package, how efficient of you!" --Londo Molari
  5. Mod Parent Up by mpapet · · Score: 5, Insightful

    I'd like to add that while the way he handled being surrounded by idiots was wrong, he was clearly surrounded by idiots.

    No documentation?
    No change control?
    No diagrams?

    What really rubs me the wrong way is how you haven't heard a single word from the admin and yet he is blamed for everything.

    I worked one place where a guy with a great deal of responsibility died. (here today dead tomorrow kind of thing) His peers blamed *everything* on him simply because they could. This sounds like the same thing.

    --
    http://www.maxineudall.com/2010/02/should-economists-be-sued-for-malpractice.html
    1. Re:Mod Parent Up by AioKits · · Score: 4, Insightful

      What really rubs me the wrong way is how you haven't heard a single word from the admin and yet he is blamed for everything.

      Well, every Stalin needs his Trotsky!

      --
      "Quote me as saying I was mis-quoted." -Groucho Marx
    2. Re:Mod Parent Up by Sobrique · · Score: 5, Insightful

      Wait, you mean blame it all on the guy who left (be it through death or a cushy new job) isn't standard practice everywhere?

    3. Re:Mod Parent Up by rickb928 · · Score: 5, Insightful

      I took a gig recovering documentation and re-establishing procedures for a great admin who died as well. He really did great docs, but no one had ever used them, and they couldn't figure out the 'copy file piopoiop.dfj to the \asic\wer\2344\sdf.msdfn folder' sort of directions.

      And the crew there immediately set to removing, replacing, and destroying all of his systems. He was a Novell hardliner (so was I), and when he was gone, his boss succumbed and the Windows bigots prevailed. Much taxpayer money was spent replacing perfectly functional systems. Mind you their clients were still running Novell, so there was some disconnect when they would get a request for support and start saying 'you have to upgrade (ha!) to Windows'. Their clients, for reasons best left undisclosed, could not upgrade. Both physically impossible and logistically impractical. Start with being 60-1600 meters below the ocean surface, and it only gets more difficult from there.

      I'm a little surprised that SF hasn't worked this out. There are plenty of outfits eager to do what is necessary, for a fee of course.

      And yes, finding a device is not impossible. Finding the connection to the network is the obvious first step. After that, well, kill it.

      Unless it's hiding. That would be unfortunate.

      ps- This guy, by many accounts, was brilliant. And a little off the wall. Goes together.

      --
      deleting the extra space after periods so i can stay relevant, yeah.
    4. Re:Mod Parent Up by OnlineAlias · · Score: 4, Insightful

      Could be an IBM 3174 like device too, running SNA. Fact is, the article and and court filings aren't clarifying any of this and leave the door open for mass amounts of conjecture and sensationalizing, both in the media and on Slashdot. Which, of course, is exactly what everyone is doing...

  6. Re:MAC search by Archangel+Michael · · Score: 5, Insightful

    I learned early on, that most people don't see the difference between a $12 hour high school geek and a $75 hr network administrator. All most people see is that both do roughly the same job and there is $63 hour difference.

    Most of the time, the $12 hr guy is doing most of the same work as the $75 hour guy. The big difference is when crap like this comes up, the $12 hour guy can spend years trying to figure out what the $75 hr guy can figure out in 5 minutes.

    Even when the $12 hr guy screws up, the response is "But he was cheaper". It is cheaper to keep a $12 hr guy trying to keep crapware off a computer, rather than a $75 hour guy who doesn't allow crapware in the first place.

    The point I'm making, is that a $75 hr guy is worth it, but only to people where time has real value. People who place no value on TIME, don't care about anything other than $ per HR

    --
    Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  7. Re:You're an 1D10T by denis-The-menace · · Score: 4, Insightful

    I wish I had mod point for you.

    Chances are that internal policies prevent the use of "hacker" tools to secure the network.

    Again, the PHBs are idiots!

    --
    Obama's legacy: (N)othing (S)ecure (A)nywhere and (T)error (S)imulation (A)dministration
  8. Re:to quote bash.org... by russotto · · Score: 4, Insightful

    2) He set all the network devices on the network to lose all info on a reboot

    I wonder if this one is just a complete misunderstanding. One article says that they were set to lose configuration files on "reset". That's pretty typical -- if you have some device you don't have the password to, you can do a full factory reset and get it back to the default password, but that also wipes the configuration files. He might have told his incompetent bosses that, and they thought he meant they'd lose the files on a reboot instead.

    Anyway, if this guy is what they're making him out to be, they need to completely wipe and reconfigure the network anyway; it's the only way to be sure he didn't leave a few presents for them.

  9. Admin code of ethics. by khasim · · Score: 4, Insightful

    What would you think of a doctor who, because some exec somewhere decided he should, pushed the WRONG medication / procedure to you?

    Where does your ethical responsibility end and the boss's desires begin?

    To me there isn't even a question. Fire me. Go ahead. I will get another job.

  10. Re:You're an 1D10T by Archangel+Michael · · Score: 4, Insightful

    Yes, both of those are true (Mac, Ping). Even NMAP responses can be spoofed. However the likelihood of all three being done is not likely. However NMAP will reveal a used IP, and a mac table somewhere will identify what port it is hanging on. Packets have to be routed to it somehow.

    And I agree with your last point. I'm a Libertarian. ;)

    --
    Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
  11. Re:I've Changed my mind. by Medievalist · · Score: 4, Insightful

    Oh yeah, let's give him a break. Oops, he's been by hit by a bus. Where's his disaster recovery plan? That's right, there isn't one.

    My bet is, it's sitting right in the middle of his old desk blotter, in a fat manila folder marked "Disaster Recovery and Service Continuity Plans". These clowns would never find it there in a million years. The infamous missing passwords are probably in a letter-size envelope in the top left desk drawer, too.

  12. Don't mod that "funny". by khasim · · Score: 5, Insightful

    It appears that the idiot "boss" is attempting to generate support for the claim that this guy is a "problem" by paying unreasonable amounts to "repair" the "damage" he did.

    It's difficult to "prove" that a guy did millions of dollars of "damage" ... without a bill for millions of dollars of "repairs".

    Any competent network admin could map out the network and document it for FAR less than the hundreds of thousands of dollars that is being thrown about.

  13. Re:Simple: by cecille · · Score: 4, Insightful

    ...you tell us, Mr. "anonymous".

    --
    ...no two people are not on fire.
  14. Re:Simple: by CrossChris · · Score: 5, Insightful

    MCSE:

    Must Consult Someone Experienced

    Minesweeper Consultant and Solitaire Expert

  15. The new WarLords by DeanFox · · Score: 4, Insightful


    I'm reminded of a conversation I had some 25 years ago with a co-worker IBM mainframe technician. IBM management was incensed that uneducated morons turning screwdrivers could make 70k a year. Back then as much as what they were paying top MBA stuff shirt types. They were on a mission to get salary levels down to "reality" paying these screwdriver wielding monkeys what they were (in their minds) really worth.

    Attitudes have changed but not a lot. 93% of companies that loose their data center for 10 days or more due to a disaster filed for bankruptcy within one year. 50% filed bankruptcy immediately (National Archives & Records Administration in Washington). One can't say the same thing about those over paid MBAs.

    It may be awhile before IT matures into a "profession" like doctor or lawyer however I personally believe we're holding the keys. The world can't function now without us.

    -[d]-

  16. Re:Simple: by ajrs · · Score: 5, Insightful

    and your not sniffing the traffic to these boxes why?

  17. Re:You're an 1D10T by gad_zuki! · · Score: 5, Insightful

    >But everyone who supports more government ought to take a look at the incompetence here.

    Im one of those crazies who doesnt support more or less government. Just better government.

  18. Re:Simple: by kimvette · · Score: 4, Insightful

    Nobody knows what the passwds are to get into them so I can't log in and find out what they do.

    1. Boot from floppy, optical media, network, etc.
    2. mount [/dev/sda1|/dev/hda1] /mnt -o rw
    3. chroot /mnt
    4. passwd root [password]
    5. ??????
    6. PROFIT!

    No yanking to do. A reboot and 5 minutes of down time. Bang. Dead. Done.

    --
    The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
  19. Re:Simple: by Firehed · · Score: 5, Insightful

    Poison gas ? You think that's all an evil supercomputer will do ? NO ! It will spontaneously develop godlike powers, take over the universe and unravel the very fabric of reality around you !

    It may also mock you with nonexistent cake.

    --
    How are sites slashdotted when nobody reads TFAs?
  20. Re:Simple: by blind+biker · · Score: 4, Insightful

    I have a huge admiration for your honesty. You are an exceptional person.

    --
    "The agriculture ministry is not in charge of Gundam" - Japanese ministry official.