Tales From the Tech Trenches
GMGruman writes "Anyone in IT has a story or two involving stupid users, crazy co-workers, kludgy technology, and airhead managers. Lisa Blackwelder has collected top tales of the tech trenches, covering user antics, office politics, and unusual technical challenges that IT pros faced (usually) with aplomb, insight, and savvy."
http://www.infoworld.com/print/146869
Because there's really no reason to post that shit on two pages to cram in more ads.
You're better off reading The Daily WTF for these types of stories. It's better written than InfoWorld could ever hope to be.
There's no -1 for "I don't get it."
This month's horror story concerns a user reporting a nasty security issue.
The user comes up to the helldesk and reports that they have a, quote, "mysterious cable" coming out the front of their computer. Given that at $company we pay a little more attention to security than, say, Gawker, one of my fellow Ops techs was dispatched to the user's desk to determine what this cable could be and why it was so mysterious.
A few minutes later, he returned, having successfully traced the mysterious cable out the front USB port all the way to the keyboard.
Upon reporting this finding, another tech asked who the user was--and then noted that she had given said keyboard to said user, who had plugged the keyboard into the USB port herself.
In Xanadu did Kubla Khan
A stately pleasure dome decree
This story isn't unique to any single one of us. Reply inanely here if you too have had a cleaning lady pop a core something (switch|router|something better) to plug in a vacuum cleaner. There's at least ten of you. Come on.
We need to give this cleaning lady (ok, or cleaning man -- no -- cleaning person) a name, like "Reboot Bertha", so that we can just call her this from now on. Alliteration counts.
slashdot: where everyone yells sarcastic metaphors to themselves to understand the issue
Several years ago, I was the Network Administrator at a medium sized manufacturing company. We had a problem with some of our employees viewing inappropriate content while at work and we didn't have a robust content filtering system at the time. So, I set up a sniffer port at our Internet ingress point and connected a laptop with Driftnet to get a real time picture, literally, of what was being accessed. One of my fellow IT workers saw what I was up to and decided to DoS me. He created an image of a Nazi Flag with my picture superimposed and the words "Network Nazi" underneath. He then uploaded the image to his personal web server with an auto-refresh script and for the next 20 minutes, the only thing I saw was the image he created / posted.
Seriously, my gripe here isn't about being spread over two pages, my gripe is that there is literally shit everywhere on that site. I have never seen such a "busy" lay out, with the facebook shit on the side and ads on the other, topped off with text ads in the middle of the article.. fuck that.
Eat sleep die
Always mount a scratch monkey.
-gz612
At one point I did tech support for a company that had inherited a block of 1 way cable modems from a company they bought. One way cable modems are the very definition of asynchronous - 56Kb upstream & 3MB downstream at the time (55:1 ratio) - as they bond a dialup connection upstream with a cable connection downstream. Not only does this compound the number of problems - all the problems of a dialup modem and the problems of a cable modem with the added joy of bonding issues - but customers were completely unable to grasp the asynchronous nature of the process.
One customer in particular was quite upset that it took so long to upload his files. I can only blame myself as I asked "What kind of files are you working with?" I then endured a 20 minute rant on how it didn't matter what kind of file it was because he had downloaded the files quickly from the newsgroups earlier and it shouldn't take hours to repost his new donkey porn videos to a different newsgroup.
Obviously, the constant barrage of stories linking back to InfoWorld here, posted always by the same employees at InfoWorld, is the result of money changing hands between InfoWorld and Slashdot. This is a slow week for news, so the story is even more tedious than normal. Still, they've already pad for the linkage, so they gotta fill it with something...
Wait... what?
Infoworld really sucks at giving Information...
It pays to be obvious, especially if you have a reputation for being subtle.
Indeed, everyone in IT has stories about how everyone except themselves are idiots.
Read Dilbert instead
Two websites I've enjoyed for tech horror stories are:
Tech Tales [http://www.techtales.com/]
Clients From Hell [http://clientsfromhell.net/]
So a company I worked for had about 80 people in manufacturing and design engineering in one building. Some electrical contractors were doing upgrades to the manufacturing area -- pretty normal. But two guys on the crew wanted to make sure the line they were working on was dead, so they went to the main breaker panel and started systematically flipping breakers to identify their circuit. They had worked their way through all the circuits powering the engineering workstations, crashing Unix machines right and left, and had started on the circuits powering the PC board stuffing robots, etc., in the manufacturing area. The breaker panel was visible in that area, so one of the manufacturing managers figured out what was happening and put a stop to it. Still, it cost us in engineering most of a day to recover.
The manufacturing VP was a cool guy... he immediately walked the whole crew out the front door and called their boss to report their firing from the job... and said the tools they left behind would be sent to them. A lucky friend of mine got to pour all their tools randomly into a moderate-sized crate and wheel it to the loading dock.
Ohh... as to your naming contest... my contribution: Rita Reboot
.... doesn't mean you should.
Best story I've got this year is attempting to help a customer repair a corrupted Exchange information store.
What we found while helping this customer run the repair tools was that they had put their Exchange databases on a software RAID1 between an internal SATA drive and an external USB drive...
All the while, the server in question already had an internal hardware RAID controller with a RAID5 with roughly 3 times available space as the size of their Exchange databases.
On top of that, their backup was a backup to disk folder, on the same software RAID1.
It appeared that the internal SATA drive, which was not properly mounted in the first place due to the fact that the server chassis was designed for hot-plug SAS on a backplane, had failed some time prior, and now the USB drive was starting to experience excessive bad blocks.
We were able to repair their database after moving it to the hardware RAID.
putting hot electronics in the fridge or freezer for breif periods does sometimes help. its just colder... its not freakin raining in the fridge...
Someone asked me how many continents they can put in their recycle bin. I told them seven.
Are there any old Mindspring folks here who have access to some of the f.sharpe's call logs? I worked in the NOC '98 thru '02 or so and heard about her a lot. After all these years I remember tales of her calls to the CSV guys. Remind and entertain me again, guys. TIA
I used to work at a shipping company. They used to provide higher volume customers with a PC that ran their shipping software. One of my responsibilities was to maintain this equipment at the customer sites (upgrade software, fix PCs, etc.). So the company assigned me to the Florida West Coast (Naples, Ft. Myers, etc.). I arrived at the facility with a list of about a dozen customers to see that week. My first task was to get the spare equipment to replace/install at customer sites. There was supposed to be a small room with all the equipment. I went to the room but it was empty. I called the home office... They said something to the effect that "It's a large room with a whole bunch of PCs, monitors, keyboards. You can't miss it." So I looked again. Even though it was a good sized warehouse, there were only four or so rooms. Nope, couldn't find all that equipment. Finally got in touch with the manager there.... Yup, my predecessor had loaded all the equipment into a truck and taken it away.
I called my office. And yeah, I figured that something suspicious had happened but I had to play it dumb (can't go around accusing someone of theft if I wasn't certain). Call went something like:
"Hey, the former admin took all the equipment away. Where did he take it?"
"What do you mean he took the equipment?"
"I understand that he loaded everything into a truck last week and drove off. Let me know where and I will see about moving it back."
"What do you mean he took the equipment?"
"The facility manager said he took the whole day loading everything up into a Ryder truck. Then he drove off."
"Where did he take them?"
"I don't know. I just got here today. "
The beauty of his move was that he maintained all the inventory... So when it came time to see how much equipment was supposed to be there, everything showed as empty or at customer sites or disposed off... A roomful of brand new equipment was marked as "Disposed" or "Sent back"...
I "graduated" from a desktop technician to a full fledged web developer quite some time back -- but yes, I do indeed have some interesting stories from back-in-the-day. One of my favorites is the exceptionally overweight guy who called me to fix a problem with his MacBook Pro after having sat on it. The odd part to me was that he somehow thought that the issue was related to software, and he didn't even admit to what he'd done until I turned over the Mac, and asked him point blank about the huge freaking dent in the bottom. Needless to say, the computer had a cracked motherboard and had to take a trip with me to the nearest Apple Retail Store.
Just this week, I watched my new boss (the helpdesk manager) flail between the delete and backspace key for 30 seconds to delete a line from a text file.
/.
She also has trouble differentiating between ProductName1 (website), ProductName2 (other website), and ProductNameAdministrator (helpdesk interface for both).
I probably don't have to post anonymously, she probably never heard of
Ok, well I have to take some blame because I was involved in this, but while working for a major retailer I was one of two engineers fixing the power going to a pair of 6509's. They had redundant power supplies, and both the backups were bad. I had sent them both back, and received the RMA units the same day. After scheduling the change, and getting all the paperwork filled out we were ready to begin. Because we anticipated issues with at least one of the units, anything in this Datacenter seemed to be cursed, we called in a proactive ticket with Cisco. As we lined up the 30 amp plug and had it seated in the plug housing (attached to a local UPS) the engineer I was working with began inserting the 20 pound power supply into the chassis.
Just as he was sliding it I noticed THE CABLE HOUSING WAS SLIDING OUT OF THE POWER SUPPLY!!! I was starting to shout for him to stop and the two exposed solder points contacted the outside of the power supply. Needless to say, milliseconds later, Sparky (who hadn't checked the screw that held the housing in place on the power supply) was cowering in the corner, the operator on duty ran in the DC and had to yell over our now popped ears what the fuck just happened. Occording to her it was a very large bang, to me it was like a lightning bolt in front of my eyes.
I was already reaching for the leather strap to yank him off it, when I saw he was on the ground and the UPS had locally blown it's fuse. Thankfully he wasn't hurt, and it only took me about 36 hours of explaining to TAC what happened to get the unit back up to 100%. Before that night I never thought I'd call and say, "The unit arc'ed out and I watched it ground through the chassis... we're gonna need some parts." From now on I write the instructions such that it's painfully fucking obvious "DON'T FLIP THE POWER TO THE ON POSITION ON THE FEED UNTIL THE UNIT IS SECURE!!!"
Sparky doesn't do IT anymore.
I watched fascinated as a desktop tech attempted to get more mileage out of a laser toner by shaking it -= then when that failed, held it over his head, stood underneath and prised it open with a screwdriver. When I finished laughing (not that the inhalation of toner is healthy) I suggested using the hottest possible water to clean the toner out of his clothes. The following week the same person rolled out a number of new PCs, and connected them to the network with 5 metre cat5 cables - when I went up to fix the alleged routing problem I found he'd used his "nouse" (and muscles) to stretch the cables. The job sheet clearly said 10m cables, guess he was just lazy. Oh, another time he "discovered" that an entire batch of new replacement hdds was dodgy - by hooking them up one-by-one, with the the drives sitting on the metal case (shorting the circuitry) and booting the test machine. Finally, in response to all the complaints his father stepped in a moved him out of Desktop support - to Project Management. No surprise that company went into receivership the following year. (I'm looking at you Adam!).
Then there was the new starter in the server room - not 15 minutes after being read the rules on *not* touching the mainframes or the tape robots ("just stick to labeling blank tapes") he climbed into a tape robot "to see how it worked". It was only luck and a lot of micro-switches that prevented the robot from tearing his head off. (there's a special place in Hell for people who "look" with their fingers). I should have seen the signs when he first tried to install Outlook on his PC - I asked wtf he was doing - he replied that the "don't fuck with the equipment" rules were only for people who weren't computer experts (he held a MSCE), and that surely everyone knew Lotus was an "end-of-life" product.... One guess who the employer was (manufacturer of said mainframes and tape robots).
Both these people are still employed (elsewhere) in the industry, the latter is considered an IT industry authority in the local PC user's group.
This lawn supervisor was out on a sprinkler maintenance job and he started working on a Findlay sprinkler head with a Langstrom 7 gangly wrench. Just then, this little apprentice leaned over and said, “You can’t work on a Findlay sprinkler head with a Langstrom 7 wrench.” Well this infuriated the supervisor, so he went and got Volume 14 of the Kinsley manual, and he reads to him and says, “The Langstrom 7 wrench can be used with the Findlay sprocket.” Just then, the little apprentice leaned over and said, “It says sprocket not socket!
Display some adaptability.
As an Intern, I was tasked with fixing the CEOs laptop....in so doing, I had to format it. Well, I backed up everything except for his Outlook email file (.pst)...and he never used to store anything on the server. It's a wonder I wasn't fired.
The first story comes from an old admin I worked with. Apparently he worked for a hospital as a helpdesk person and received a call from one of the higher ups. The issue expressed over the phone was a request for a larger mouse pad as he kept running out of room. When he heard this, he shook his head and decided to take a walk up to the office to find out what was going on, When he got there and looked in the door, he had to turn around and walk away, because he was laughing so hard. The executive would use his mouse and hit the end of the mouse pad. Instead of picking the mouse up, pulling it back and then moving the mouse again, he would pick the mouse up, slide the mouse pad forward, set the mouse down and then move it. Anytime it hit the edge of the pad, he would do the same thing.
The next story comes from my old beowulf days. I was working for a company as an admin. They staff were programmers that designed DSL Modem Processors and the software that run on them. One day another admin and myself decided to play a prank on the staff. I remebered a tool that we used when creating beowulf clusters that would allow me to issue a command to multiple machines at once. We used it for configuration. Well, it is also good for ejecting cd-roms. We loaded up an inventory of machines and typed eject cdrom. Its amazing the sound of 35-40 cdroms drives opening at the same time. It's even more funny when you hear the expressions of 35-40 staffers saying "WTF?"
Good Times
One time, I got a call because a computer was ticking loudly, like someone stuck an alarm clock in it and had to go down and deal with that. It was quite tempting to say that was exactly what it sounded like, but it was just a defective CPU fan Dell gave us.
It was fixed easily enough, but luckily the user didn't panic at hearing the traditional "this will blow you away" noise the computer was making.
I got a computer once where a CD was jammed between two CD drives, similar to what you see in the summary pic. A 3-year-old girl did it.
"When information is power, privacy is freedom" - Jah-Wren Ryel
A client of a previous consultant job owned a wild life rescue. One day she brought in her laptop that a Raccoon had used as a litter box (It was a Gateway, so not far off.) Rubber gloves, masks and moving to the companies attached garage with a card table was required to get the data off the HD and into the new one she purchased. The garage reeked of Raccoon musk for quite a while.
See subject line, because you're not fooling anyone with your
"DOWN MOD SQUAD" "tactics", LOL:
---
http://www.youtube.com/watch?v=kd85Qim_Z6A
I can see it now: The white guy @ the start (Michael Cole) is getting loaded with malware and tracking him online, & he's SICK OF IT. He runs into the black guy (Clarence Williams III), who also tells him he's feeling the same & sick of it. Then, Michael Cole says "She's feeling that way too" as Peggy Lipton comes rushing onto the scene. They tell one another about HOSTS files & other methods to protect against that... & who comes "popping outta the woodwork"? CAPT. TROLL! Complete w/ his "down moderation" (as TIGE ANDREWS comes into the scene, lol)... he needs a "thought balloon" that says:
"I can't let others know how to stop my bogus machinations in maliciously scripted websites &/or adbanners. I have to either TROLL THIS, or down mod it and run!"
LMAO!
---
Downmods?
Hey - they're not very useful vs. facts!
APK
P.S.=> So, see my subject line & keep burning up your mod points boys... it's YOUR loss! apk