Slashdot Mirror


The Oft Frustrating Job of a Sysadmin

I_Love_Pocky! writes "Sysadmin Co. is a hilarious site built by some sysadmins at an ISP to help them vent their frustrations with dealing with non-tech types. This site is gives a hilarious picture of the daily frustrations of dealing with the inept. I am interested to see if these stories strike a chord with other admins out there."

34 of 588 comments (clear)

  1. My favorite statement came from a 'sysadmin' by Tangential · · Score: 5, Funny

    I'll never forget, he said "There was no way to know that the backups were failing without looking at the log file." This statement was made 17 months after the backups stopped working....

    --
    Suppose you were an idiot. And suppose you were a member of congress. But then I repeat myself. -- Mark Twain
    1. Re:My favorite statement came from a 'sysadmin' by sharkey · · Score: 5, Funny
      "There was no way to know that the backups were failing without looking at the log file."

      From my efforts to get FreeS/WAN talking to a Cicso VPN concentrator:

      Tech at other end: "Well, I'd have to copy and paste that section of log to send it to you."
      --Trying to explain why it was so hard to send me the bit of his log that would tell me what was fucked up on his end.--

      --

      --
      "Outlook not so good." That magic 8-ball knows everything! I'll ask about Exchange Server next.
    2. Re:My favorite statement came from a 'sysadmin' by Junta · · Score: 5, Interesting

      My favorite conversation I had as a sysadmin highly frustrated years ago. We had some old SunOS systems and newer, but still old HP-UX systems that came in and was trying to get things to work at least relatively seamlessly, and for some reason or another something wasn't working right (was many years ago, have no idea, probably an nfs issue/nis configuration). Anyway, so one non-technical user was there as I was trying to get some basic, critical functionality restored. She was curious and asked:
      'What's wrong with it, what are you trying to fix?'
      My response: 'You see, our old network smokes crack, and these new systems.... well they smoke crack too, but it's different crack and they don't seem to be capatible crack'
      She gota tad angry and obviously felt insulted by my talking down (probably thought I was talking that way because she was a girl) and said 'I can handle a more technical explanation than that!'
      My honest response: "Well *I* can't"
      At which point she understood and laughed rather than be angry.

      Another one of my past stories, I was working with this contractor once and he was charged with the task of configuring a new HP-UX server that had been ordered. He hooks it up to the network alongside the main nfs/nis HP-UX server of the company, and strolls back to his desk and telnets into the IP he thought he was assigning. Suddenly he thinks 'hmmm.... the hostname of this new box happens to be the same as our main server... better change that.... wow, the IP it will enact in a minute too, that is *really* weird, well, better change, reboot and.....' Suddenly, across the company systems hung as the NIS/NFS server moved. The contractor had no idea what was happening until someone else took a look...

      --
      XML is like violence. If it doesn't solve the problem, use more.
    3. Re:My favorite statement came from a 'sysadmin' by Anonymous Coward · · Score: 5, Funny

      How about this one: The people that I was councelling regarding a new installation said that they could save a lot of money by installing each XP CD on two computers. When I asked why they thought they could do this, they stated that the CD said that it could be done. After much gentle prodding, they gave the reason for this, the package said, "for 1-2 CPUs". I explained that this meant motherboards that supported up to two CPUs. They looked at me like I was from mars.

      These strange discussions went on and on with various subjects. I mean, they really didn't have a clue and would not accept any other view. In the end they chose not to work with me and choose someone who shared their level of "understanding".
      (i.e their server has no firewall, backups aren't verified, no intrusion detection, no documentation, etc..)

    4. Re:My favorite statement came from a 'sysadmin' by Junta · · Score: 5, Interesting

      Story from my previous doomed job:

      After I had been working there a while and mentioned time and time again their infrastructure *needed* a more controlled storage strategy (as it was, critical data was spread across many non-redundant systems (whichever happened to have free space at the time), and added drives ad-hoc to whatever was there. When time was up on the current storage and they were just about to order an extra 40 GB disk to slap in an Ultra 10 to bolster things, one drive crashed hard and took out critically important data. So I'm finally asked to, on a shoestring budget, give them a decent file server with some redundancy. I price out a PC system to put linux on and 6 cheap IDE disks and three IDE controllers, all new, warrantied parts that would interoperate in a standard way such that any one component failure would leave an easily recoverable situation, even if not necessarily highly available (even if totally destroyed, a tape unit would at least finally be effective being attached to more than 1% of the companies date).

      After viewing this, the guy actually making the purchase says 'IDE is not enterprise quality! You can get SCSI storage on a shoestring budget!' and proceeds to acquire a rack-mountable, 14-or-so hotswap SCSI enclosure with 18 GB discs...from some random eBayer, no warranty, no service, no promises, and blows more than the entire budget I was told to go with on *just* a hardware RAID controller. After a week of them using it strongly against my warnings, the whole thing goes down unrecoverably bad... turns out the SCSI enclosure had a malfunctioning backplane and had been corrupting data all the while....

      It turned out that before I was around they had a Maxtor IDE-based NAS with two drives per chain and running Windows 2000. One drive went bad and the system went down hard as the other drive on the chain was unreachable. Though all data was recovered when Maxtor sent a drive 2 days later (they didn't want to run non-redundant or with unsupported IDE disks for fear of losing something without recourse.. understandable at least) and so the business guy had learned IDE==bad, lose data.
      I told him that in this case, it would be one drive per channel, and in the event of failure any ol drive from the local Best Buy would do and he wouldn't have to wait days for a replacement, but that whole job was an exercise of great accountibilty with zero authority to do anything about anything...

      Almost every job before and the job I have held since has been infinitely better.

      --
      XML is like violence. If it doesn't solve the problem, use more.
  2. Doesn't really strike a chord with me, nope. by RLiegh · · Score: 5, Insightful

    Then again, I have found that treating my userbase as people, and not as trained monkeys, tends to have better results than trying to be mister 31337 BOFH.

    1. Re:Doesn't really strike a chord with me, nope. by zaxus · · Score: 5, Funny

      That's just crazy talk. You mean they're people?

      --
      /. zen: Imagine a Beowulf cluster of Beowulf clusters...
    2. Re:Doesn't really strike a chord with me, nope. by ender81b · · Score: 5, Funny

      Agreed. I'm sitting at work right now doing tech support... on the line with some guy who has problems figuring out what exactly a "browser" is.

      People? Meatbags. Meatbags I say.

    3. Re:Doesn't really strike a chord with me, nope. by Ieshan · · Score: 5, Insightful

      Many sys-admins don't realize that the people they work for often have technical skills in other areas and simply don't have time to deal with computers.

      It's sort of like being a mechanic. People do all sorts of stupid shit with their cars, but that doesn't make them stupid people. It just means they have little technical expertise dealing with cars.

      That said, I *do* tend to have little patience for people who won't read a manual. I tend to take a very DIY attitude towards things - that's the most frustrating part of trying to explain computers to others. If people would read the document that read 'README' or read the error message instead of panicking when one occured, 95% of all computer problems would be fixed instantly.

    4. Re:Doesn't really strike a chord with me, nope. by dspeyer · · Score: 5, Insightful
      Some are, some aren't. If you need to check, carry a gom jabbar.

      Seriously, though, there's an immense difference depending on who you work with. I have two system administration jobs, one with an astrophysics project and one with our campus "student union", which means the people who organize clubs and activities. The astrophysicists I tell the whole truth, and they can apply what I've told them. The buerocrats and student organizers are hopeless: I fix their computers and try to block them technologically from doing stupid things.

      So I guess the message is don't assume all users are the same -- take a minute to categorize them (it doesn't take two).

  3. My Favorite by Blair16 · · Score: 5, Funny

    was a lady running a Mandrake system asking me about saving files to a disk. She was having troubles and thought that maybe automount wasn't working. I went over there, put in a disk, and copied the file.
    Her problem - she hadn't put a disk in the drive.

    --

    Chaos will always win out over order because chaos is more organized
    1. Re:My Favorite by OneIsNotPrime · · Score: 5, Interesting

      True story.

      My coworker mentioned to his boss that he could tell every time she got email because her system beeped. She said she'd like to turn it off, but didn't know how. He proposed just muting the volume. She complained "But then I wouldn't be able to hear the clicky sound of the keys when I'm typing." He gave her a blank stare, trying to figure out if she was joking or not, then she put her ear up to the keys and started pressing them. "Yeah, I couldn't hear them" she affirmed, then started clicking the mouse "or the clicking sound of the mouse."

      This is one of our lead programmers by the way. *sigh*

      --

      ---

      WARNING:Slashdot karma not redeemable in the afterlife.

  4. Site died already. by iridiumz0r · · Score: 5, Informative

    Archive.org mirror.... http://web.archive.org/web/20030714083852/www.sysa dminco.com/main.php Seems to still work, haven't tried loads.

  5. don't you find this ironic ? by Kilka · · Score: 5, Funny

    I find it funny that a site designed by a bunch of sysadmins to vent their frustrations will likely be very frustrated when /. hammers it into the ground.

    This post have been here for less that 5 mins and i'm already getting mysql_connect errors!

    -Kilka

    --
    If we don't believe in freedom of expression for people we despise, we don't believe in it at all. -Chomsky
  6. hmmph by SQLz · · Score: 5, Insightful

    I worked support for a long time. I don't think the users are inept...I think they just have other interests besides computers. I mean, if a brain surgeon or nobel prize winner calls for help on setting up dialup networking, are they a idiot user? I don't see doctors making websites about what idiots we are when we call them for medical advice.

    1. Re:hmmph by ottffssent · · Score: 5, Funny

      The reason doctors don't make websites complaining that I'm an idiot surgeon is because they're idiot users.

  7. Funny voicemail we got at work by bdigit · · Score: 5, Funny

    I am a student working at a helpdesk at a university, one day we came in to a voicemail from a user where they apparently thought they hung up the phone but they hit the 3 way button and well ill let you guys listen for yourselves.

    http://s.bouncybouncy.net/call/

  8. rinkworks-- better material by way2trivial · · Score: 5, Informative
    http://rinkworks.com/stupid/

    good for a larf...

    --
    every day http://en.wikipedia.org/wiki/Special:Random
  9. hilarious? by unknown_host · · Score: 5, Funny
  10. Borrowed from another AC post by Anonymous Coward · · Score: 5, Funny

    That reminds me of this comment from earlier today, which gives the top 10 reasons for committing seppuku (a Japanese form of ritual suicide by disembowelment)

    -----
    Here are the top 10 reasons:

    10) You've just been ordered to migrate from sendmail to Exchange server.

    9) Your boss, let's just call him Bill, insists upon being given root priviledges, in spite of the fact that he constantly breaks things even with mere user priviledges.

    8) Your boss won't let you filter out .vbs & .exe attachments at the mail server because he is an amature (read: terrible) coder. Moreover, his amature programs cause as much if not more trouble than the virus-laden attachments he keeps opening. He also has crazy ideas about putting "stamps" on email.

    7) You are told by your boss, who (mis)read a computer security advisory to put the company webserver (which handles online sales) on a non-standard port "so the hackers won't be able to mess with it."

    6) Your boss expects you to find a way to make your Solaris servers, with tons of ancient, crufty legacy code which is vital to the company, run ASP pages just so they can use (read: justify the rediculous expense of) some crappy B2B application they bought without consulting IT. Preferably sometime next week.

    5) Your boss thinks that some 'internet accelerator' software (read: spyware) should be made mandatory for all employees to improve productivity.

    4) Your "security policy" is more like a list of who to blame for what.

    3) Your boss is negotiating a SCO IP license, since "any publicity is good publicity."

    2) Your boss thinks you should be more thankful, because the management is so "IT-savvy" and always ready to help you out.

    1) You ignore all this bad advice, pretend you took it anyway (he'll never actually know...), and waste your time posting on Slashdot instead of working.

    1. Re:Borrowed from another AC post by bob670 · · Score: 5, Funny
      "4) Your "security policy" is more like a list of who to blame for what."

      I've never seen you at the office, where do you sit?

  11. Humor in an interview by NanoGator · · Score: 5, Funny

    I interviewed for a web-admin job a few years ago. They asked me "How would you troubleshoot a blue screen of death?" With a smile on my face, I replied "I'd press F1 and ask Clippy!" Ah we all had a good chuckle at that. Heh. Didn't get the job, though.

    --
    "Derp de derp."
  12. Latest bit... by Vrallis · · Score: 5, Funny

    Not your typical Sysadmin story...

    I work for a large auto parts retailer (nope, not Auto Zone!).

    Each of our stores has a Linux system in it, using Comtrol serial boards to run the serial terminal and printers in the store.

    One of our stores decided to do some rearranging, and wanted to move the main counter a few inches. The counter isn't bolted to the floor, but it does run the full width of the store, and is pretty much permanently wired for electricity and serial connections where it is--it's not meant to be moved.

    So, what did the store do? They moved the counter. With everything on it. With all the terminals and printers on said counter plugged in. And turned on.

    The employees heard a few 'pop's and looked up to see smoke coming from all the terminals.

    The best we can figure is the main power line running into the counter was punctured or otherwise shorted, shorting hot to either ground or neutral. Naturally, the terminals weren't on any sort of surge protectors. I doubt this would have helped, though, unless they had good Triplite or another good name-brand surge protector on it (which won't happen--too expensive--yep, the usual story).

    The incident didn't just destroy the terminals on the counter, though! It made it's way through the serial lines and destroyed every piece of serial-connected equipment in the entire store.

    The serial card looks like somebody took a blow-torch to it. I really wish I had a picture of it to post here, but I haven't taken one yet (it's hanging on our 'wall of shame' at the moment).

    Amazingly--somehow--the PC is fine. I've had it running stress-tests for 3 weeks now, with no problems. There are scorch marks around the PCI connector and in the bottom of the case. Most of the ICs on the serial board were reduced to nothing but ashes instantly--the rest blew into pieces.

  13. IMHO by bruthasj · · Score: 5, Insightful

    The most dangerous sysadmin is the one who believes that he's dealing with inept people when the real ineptness is found within. Whole corporate IT policies are dictated by these people *all* the time.

  14. My Favourite story by schon · · Score: 5, Funny

    This was several years ago - before DHCP. As sysadmin, I kept the list of IP addresses assigned to the computers.

    Newbie tech, right out of school (I'll call him 'D.') comes up to me, while I'm in the middle of something.. he says "I'm working on the machine in shipping, and I need its' IP address."

    I say "no problem", point to a piece of paper, and say "they're all on that piece of paper". He takes the piece of paper, copies down the number, and goes away.

    A few minutes later, he comes back, and says "that must be wrong - it tells me that it's in use."

    I tell him "that's weird - I'll come take a look at it in a few minutes."

    So I finish what I'm working on, and go to shipping.. I ask "D. said there was a problem with your machine." They shrug, and said "it's working OK right now." Just to be sure, I take a look at it, and the IP address is correct, and the machine is working fine, so I go back to my desk.

    Two hours later, D. comes back to my desk and asks if I'm done yet.. I tell him I went to the shipping computer, and it was working fine.

    He tells me "No, I'm at my bench, setting up a new system for them, and when I enter the IP address and connect to the network, it tells me that the IP address is in use."

    I guess he skipped the class where they talked about IP addresses having to be unique.

  15. Another story; and programmers vs. techs by Atario · · Score: 5, Interesting

    At a government office where I worked once (best places to get this kind of story, near as I can tell), one of the techs (who I sat near) came in and told everyone to gather 'round. He said he just got back from a half-hour conversation which consisted of him explaining to a user why the print-screened copy of an application window she had put in Word wouldn't respond to its buttons being clicked. "It's like a calculator and a picture of a calculator. You can't press the buttons in the picture and expect it to work, can you?" A larf was had by all.

    Now, being a technically-inclined programmer, I rarely have reason to deal with techs. Most of my problems, I successfully deal with myself. Therefore, unfortunately, most of the interactions tend to be about disruptive hardware upgrades or else special handling for me because I need more access than is standard in the organization. At one place I worked, not only could you not install your own software by default, but in fact had no access at all (much less write access) to most of your own C: drive. They give me more permissions grudgingly and eye me with suspicion, and even then only after being so ordered by a mutual superior. So naturally I tend to see them as fascist policy-drones. Too bad, that; we'd probably be good geeky buddies otherwise.

    --
    "A great democracy must be progressive or it will soon cease to be a great democracy." --Theodore Roosevelt
    1. Re:Another story; and programmers vs. techs by Xawen · · Score: 5, Insightful

      At one place I worked, not only could you not install your own software by default, but in fact had no access at all (much less write access) to most of your own C: drive.

      So, just a matter of curiosity, you think that it would be better to run an environment where any user can install unlicenesed software and delete critical system files? How about get infected with a virus, and due to thier heightened access it's able to delete the OS. To be perfectly honest in today's IT world, you can't trust the people using the systems with any sort of access that can affect the system itself.

      Unfortunately that goes double for the "technically inclined" users. Sure, you may have have a top notch, dual processor, 5 gig ram desktop at home running 200 different operating systems on a souped up wireless network but you don't know THIS environement. Things are different in a large network environment, and if it's not your job to run it, you probably don't know all the rules. I have had to spend more hours than I care to count fixing something some developer broke because his manaager forced us to give him administrative access on his machine. It's rarely that the person isn't technically capable, but that they don't know how our systems are set up. It's easy for someone to make a mistake when they have access to things they don't understand.

      It is by no means an insult to your technical abilities to lock your machine down. It is simply the only way for the sysadmins that are responsible for your system to ensure that it's working properly.

      I'm sorry if I seem a little testy, but I just spent 2 weeks screwing with virus damage because certain users have access to things they don't need. This post just hit a well timed nerve...

    2. Re:Another story; and programmers vs. techs by djp928 · · Score: 5, Insightful
      I empathize with those "fascist policy-drones", because they are as anal as they are for good reason. Who gets blamed if a system crashes? Not the programmer who wrote a bad application that systematically allocated every available byte of RAM. Not the dumb-ass manager who opened an email attachment and unleashed a virus on the company. The sysadmin gets blamed.

      Programmers as a general rule think they're the shit, even when they're not. Just like in my world of system administration there are five "paper" MCSEs and CCNAs for every one real system or network admin, there's five dumbshit programmers who only got into the field for the money for every true geek programmer. And these dumbshits think that since they spent three or four years in some school learning how to program, they're naturally qualified to do *my* job as well.

      The fact that you find it strange that you didn't have access to your own C: drive is a typical reaction, but there's a good reason for it. That's not your C: drive. That's my C: drive. Who has to rebuild it if you fuck it up? Who has to troubleshoot it when your shit stops working on you and you call for support? Who gets blamed for you not being able to meet your deadline because your computer mysteriously crashed? Me. The sysadmin. I do. I'll get blamed even if I can prove you intentionally deleted the kernel just to keep from having to work towards your own unreasonable deadline, because they'll blame me for giving you improper access against company policy, even though I did it to you as a favor because you claimed you needed that access in order to meet your important deadline.

      As a general rule, most sysadmins will give you only the bare minimum level of access you need to do your job. And if it's at all possible to get away with giving you less than that, we'll do it. We don't do it because we're facist rules nazis. We do it because too often it's our own ass if we don't. The last virus our company got was brought in by the development team, because we trusted them to know how to install virus detection software on their systems and know how to update the .dat files regularly, and it turned out that trust was severely misplaced. Who got blamed? We did. Not the ignorant programmers. We got the blame for not taking care of the systems we were told we could trust them to maintain.

      So, yeah, if you don't have access to your C: drive, it's because the sysadmin doesn't trust you. But don't take it personally. He/she doesn't trust anybody. There may yet be hope. If you can prove your geekdom sufficiently to your local SA, you can usually earn some trust that way. We may be a totally paranoid lot, but we know true brothers and sisters when we meet them. If you can earn our trust we can usually see our way clear to bending the rules for you a few times. But don't cross us. The first time you make your SA work all weekend to fix up a mess you made of a server due to your code running at some level of access you shouldn't have been able to run it at in the first place, you'll get shitlisted, and good luck getting back in his/her good graces after that.

      Then again, maybe your entire systems admin staff are a bunch of paper admins and aren't true geeks at all. If so, I'm sorry. There's not much you can do.

      -- Dave

    3. Re:Another story; and programmers vs. techs by Anonymous Coward · · Score: 5, Insightful
      You are making the incredibly naive assumption that if you successfully lock everything down and minimize your workload, that you are doing a good job. Guess what? Your job is not the only job in the organization.

      The pit crew of an auto racing team runs around fixing things. Abused things, damage that could be avoided if they took the keys away from the drivers. But, guess what? That's their job, fixing things, and its essential to the success of the team.

    4. Re:Another story; and programmers vs. techs by gcaseye6677 · · Score: 5, Insightful

      I've seen this troll before on a related topic. What this 'admin' fails to realize is that the company computer network does not exist for the purpose of providing system administrators with power trips and an empire to control. It exists to provide value to the company. How much value does it provide when programmers need to ask the network monkey to come and change a setting so they can compile something? How much productivity is lost because people don't have the ability to install so much as a new font? If you can't run a network where you have a centrally monitored anti virus and firewall system along with a good data backup/recovery methodology, which does not require keeping the system completely locked down from everyone, then find a new job. Most admins for companies large and small have this figured out. Why don't you?

    5. Re:Another story; and programmers vs. techs by haystor · · Score: 5, Interesting

      I work somewhere that I get the same computer as everyone else.

      I have a 1.7GHz P4, with 256M of ram, and a 17" (non-flat screen) monitor.

      Obviously the ram is a sore spot since I have to run win2k. But it gets worse. I have to run Lotus Notes. I'm developing for Weblogic. If I let everything else page out I can get an IDE working.

      When I start up my computer and do nothing but open up Task Manager, there are 42 processes running. About 6 of them have something to do with virus scanning. The scanners have to run every day, during the day since the computers must be turned off when not in use. Today, McAfee's virus scanner logged 70 minutes of cpu time out of the mere 8 hours I was logged on. This was a good day though, and sometimes it uses up to 3 hours of cpu time.

      I have to put up with all of these things because everyone must be treated the same according to the IT rules. The way I see it, I get work done in spite of their services, not through the aid of their services.

      Just once I'd like to be at the company that expects and trusts its employees to do their jobs.

      --
      t
  16. Symmetrical by dmaxwell · · Score: 5, Funny

    You've just been ordered to migrate from sendmail to Exchange server.

    That one works in either direction. I'd regard the nasty things like nitro myself. If it's working doooon't screw with it. Exchange and Sendmail? That's like trying to choose between a root canal and a rectal exam.

    1. Re:Symmetrical by Throtex · · Score: 5, Funny

      However, you do get to keep your dignity when you have a root canal.

  17. What's that noise mean? by unixdad · · Score: 5, Funny

    One of my own favorite sysadmin stories comes from when I was doing support in a General's staff office. The user had been having problems with her computer so I had the computer unplugged while I had it opened and was replacing the modem.

    In the middle of the procedure, a device on the desk next to us starting this warbling noise-- user jumps a little bit and says "What does that mean?".

    "Well, seeing as that device is your phone, I think it means that someone is calling you."