Slashdot Mirror
Top 10 System Administrator Truths
Vo0k writes "What are your top ten system administrator truths? We all know them already, but it's still fun re-telling them. Stuff like "90% of all hardware-related problems come from loose connectors", even though you already know it's true, may save you from replacing the "faulty" motherboard if you recall it at the right time."
... are operator errors. But you can't tell the operator that.
australian project gutenberg is better than the original.
---------------
Getting Help
------------
Other People
------------
Windows in 6 Bytes (IA-32) : 90 90 90 90 CD 19
Never post stupid top ten administrator lists on Slashdot, lest I have to spend my time restoring a web server from backup.
the future is but past forgotten
... even though it's better than it used to be, registry corruption is still the number one cause of boot failures in Windows XP. And the contents ntbtlog.txt and the Recovery Console are still horribly inadequate tools for fixing it...
Athletic Scholarships to universities make as much sense as academic scholarships to sports teams.
The solution to the problem is almost always simpler than you think. You can often cause more problems trying to fix a problem then the original problem itself. When you try one thing and it doesn't work, reverse the changes before trying the next thing and document each change.
Bradley Holt
Add this to the list: posting on Slashdot immensely reduces productivity
Bury me in mashed potatoes.
When all else fails, reboot. If it still fails, blame the user.
He who knows best knows how little he knows. - Thomas Jefferson
I for one welcome our new... wait... nevermind... All hail sysadmins!
The 'Net is a waste of time, and that's exactly what's right about it. - William Gibson
1. Never bother with Top 10 lists
this comment is provided "as is" and without any express or implied legibility or congruity [...]
IT'S NOT A CUPHOLDER!!
"Destroy science and religion. Science would re-emerge exactly the same; but not religion." - Penn Jillette, paraphrased
Sorry, but that list was pretty lame. It's like the writer couldn't decide whether to be funny, or actually helpful. Ah, whatever, at least it gave me something to do for 5 minutes.
Clearly I forgot to equip my +5 Codpiece of Karma.
Always engage the user when troubleshooting; helps locate problems faster and makes the job a bit more human.
The weakest link(s) in your security is/are human.
Unless you're using Windows, that is, since then Windows is a giant weak link, which is in turn made up of lots of more weak links, e.g. the user and the programs.
Flamebait perhaps, but yet so true.
[sig]
http://www.misterorange.com.nyud.net:8090/2005/12/ top-10-system-administrator-truths.html
v4sw6PU$hw6ln6pr4F$ck 4/6$ma3+6u7LNS$w2m4l7U$i2e4+7en6a2X h
PEBKAC
--Keeping the flame wars alive, one post at a time
User passwords are easy to guess, but hard for users to remember.
(Christ. The article is Slashdotted already)
This is my opinion. To make sure you don't steal it, it's covered by the DMCA.
1) Never believe anything anyone tells you: always test for yourself.
2) Always ask the dumb questions: is it switched on?
3) Reboot cures most things EXCEPT rm -r * when logged in as root
After that, things could get tricky.
I had a boss once who didn't lie to me but at the same time wouldn't follow my instructions when I had to help him over the phone. I'd tell him to do one thing. He'd do something else and then ask me what to do next. I'd tell him to do what I told him to do in the first place. After 3 or 4 tries, he'd actually do what I told him to do and his problem was usually solved.
Most users should not being allowed to operate computers, let alone drive cars. Sysadmins need to learn who these people are and minimize the damage they cause. I suggest randomly changing their password every day until they quit in frustration.
GetOuttaMySpace - The Anti-Social Network
ian
There's two ends to every cable!
i once did a simple pc build for an office, put her together, got lights but no post screen, i was stumped for 2 hours onliey to relize it was gounding out becuase like an idoit i didnt put the plastic tops on the screws that held the mobo to the case.
I read your email
-You shall be very pessimistic
-Make sure you can leave exactly like it was before you touched it.
-Dont fix what aint broken.
-Start from a known state of the system (switch off - switch on).
-Even you are genius level techie, follow the manual, RTFM.
-Dont reinvent the wheel. Compare with something thats working.
-Cables are not perfect. If something doesnt connect, check lower levels first.
-If its there, ther must be a reason. Never ever delete anything. Rename instead.
-You memory is not infinite. Write what you do.
Don't plug a keyboard-wedge barcode scanner into a mouse port. (Which can be extrapolated into: If something's not working, make sure it's plugged into the right port.)
What I'm listening to now on Pandora...
from bending them around and whatnot, they develop breaks that can get pushed back together. This is what causes the problem to be intermittent. The cable 'is' bad, not going bad. People need to be more careful in wrapping their cords up. There should be a little bit of slack in the loops or else the slightest bit of pressure will cause them to develop a break.
Rule 1. They lie. End users often tell you what they think you want to hear. When asking a question you should use terms like. What does it say? vs Does it say this?
Rule 2. They don't know they are lying.
Rule 3. Sometimes they are telling the truth. Yes sometimes what you think is impossible really is happening or looks like it is happening.
See my blog http://ilovecookes.blogspot.com/ for light hearted technical information.
User: It doesn't work
Tech: Did you try?
User: No, but...
Tech: Try it!
Be a real patriot: Question authority. Think for yourself. Formulate your own conclusions.
Unless you lack plans for the weekend.
"I'd rather be a lightning rod than a seismometer." -Ken Kesey
Never put the screws back in the case until you've tested your new hardware is working.
It's a Unix system - I know this.
It's interesting how certain things don't change despite decades of technology advancements.
--
Evan
"$30 for the One True Ring. $10 each additional ring!" -- JRR "Bob" Tolkien
Maybe for a PC, but never a server.
:-P
When I started working at my job, we had serveral servers that would reboot on a cron for the sole reason that someone was too lazy to figure out the problem. We eliminated all but one of these reboots, mainly because we don't care about the last one.
My holy grail would have to be strace/truss/tusk. I would take that tool over reboot any day. It doesn't always fix the problem, but at least you will know what it is, instead of rebooting like a coward.
v4sw6PU$hw6ln6pr4F$ck 4/6$ma3+6u7LNS$w2m4l7U$i2e4+7en6a2X h
HPs Jetdirect cards have a pretty solid reputation of failing every few years
Is this really the case? We had several JetDirect enabled PCs at my former place of work and almost none of them had a card failure. We even had a few extra cards just in case. Several of the printers were actually quite old even. The biggest problem we had was with only HP-5P (I think that is the number). Some users departments did not have the money to replace those crappy old printers. On a bit of an aside, we had several JetDirect "boxes" (the external box that connected the printer port to ethernet) that were working great. I believe most everyone in the IT staff had one at home for their printers.
No One Ever Got Fired For Buying Microsoft.
Not really true. There are some shops so enamored with Novell (mostly because of bosses stuck in the stoneage) that the idea of purchasing Exchange or using a full out ActiveDirectory system with a Windows only network storage share were unheard of. I once again reference my previous job.
Not too bad of a list overall. Most of the items are right, and it is quite true. To be honest, the places I have worked there were really only a handful of problem employees, and most of them got handled directly by our SysAdmin or the head of IT because no wanted to worry about what lie they may come up with about the work we were doing.
"Some days you just can't get rid of a bomb."
#4: When you're completely stumped, make sure there isn't something inside gunking up the works.
Huh? Maybe I'm just inexperienced, but what can a computer full of dust do to make it not work? I have only seen one instance of this. A fan was clogged and stopped working and the chip under it overheated and died. Is there anything else besides this type of thing dust can cause? The only moving parts in a computer that aren't contained are fans.
Seriously, the "90% of all hardware-related problems come from loose connectors" bit is found nowhere in the article or on google for that matter...
Number One: You will die alone.
Information wants to be anthropomorphized.
Self awareness - try it!
If your job requires you to wear a name tag, carry a gun, or administer a Windows system, you have made a serious vocational error.
-- oh.... so..... sleeeeeepy.
In my Tech Support experience, I have found only three basic rules.
Rule number 1. People are stupid. This one is true of all people. Tech support, highways, shopping, whatever. This rule has been extended to cover just about any stupid thing that anyone does.
"Why did that guy just..."
"Rule number 1."
"Did she think she could get away with that?"
"Rule number 1."
Rule number 2. People lie.
Me: "Has the computer been restarted since the problem started?"
Them: "Yes..."
Me: "OK. Let's try restarting the computer now and see what happens."
Them: "What do you mean by restart?"
And when you add 1 and 2 together, you get 3. Sometimes, people are so stupid, they don't know that they're lying. You know these people. They're the ones who have "Windows 2000 XP" or "2000 ME." They're the people for whom "Nothing happens when I try to check my email. Nothing! Just this error message..." Not realizing that the error message is *exactly* what I was looking for. An error message is *not* nothing. Grr.
There is a fourth rule that also shows up from time to time:
Rule number 4. No good deed goes unpunished.
In the famous words of the leader of the Uruk Hai from his battle call at Helm's Deep in The Two Towers: "Grr."
It doesn't matter whether I'm giving or requesting tech support, the number one rule is that the person on the other end of the telephone is an idiot.
The next Cmdr Taco duplicate will be ready soon, but subscribers can beat the rush and see it early!
Don't get linked to by Slashdot!
None of the other nine truths will save your server!
Richard Steven Hack - This sig is TOO GODDAMN SHORT TO DO ANYTHING USEFUL WITH! MORONS!
Check the most obvious first!!!
...
... because this rule got violated. When you think "They cannot be that stupid"... think again and check, because yes, they can be that stupid.
If the screen doesn't work, is it plugged in? Is the power on?
If the laptop doesn't print, is it connected to the network or printer?
If your freshly installed HD is not detected, did you plug in the power cable?
I cannot imagine how much money is already wasted on transportation, man-hours, phone bills,
int main(void) {while(1) fork(); return 0;}
If the acronym contains an F, don't ask what it stands for.
Support the FairTax
Problem
Exists
Between
Kyboard
And
Chair
Top 10 lists are great. Is there any easier and faster way to write pseudo-quality content in order to draw traffic?
[sig]
One of my big truths, set standards!
I've worked in two kinds of places, ones where they set (and stick to) standards and ones that don't. Every place that doesn't use or doesn't stick to standards has always been an experience in wasted time, confusion, and lots of bugs. Those that do can seem like you're always being nagged but in the end you find things work as expected, code is far easier to manage (especially when it is someone else's), and you aren't always having to reinvent the wheel (i.e. figuring out how to fix a subtle bug again because the solution was never written down the first time).
It sounds simple but it takes discipline at all levels. Even something as documenting what you did afterwards and putting it in an orderly file system can make a huge difference but how many people bother to do it? Managers and fellow developers have to crack the whip and keep people from trying to cut corners.
Standards should be open to some change and can be bent but there has to be a very good defendable reason for it.
It is by the juice of the coffee bean that thoughts acquire speed, the teeth acquire stains. The stains become a warning
1. 90% of problems are user error.
2. The rest are us trying to fix the the first.
3. "Your mouse doesn't work because it has NO BALL!!" (I work in a school)
4. Q: What where you doing when the problem occured? A: Nothing, meaning organising the windows folder.
5. Q: Did you try to fix it yourself? A: No. Q: Why is the gaffa tape involved then?
6. Our server doesn't forget your password's you did.
7. If you save over 30gb onto your desktop don't ask me why your profile takes 10 min's to load.
8. Your mouse is moveing on its own because i have taken controll out of your silly hands.
9. Have you checked to see if your mouse/keyboard/screen/LAN/printer/Random piece of hardware is plugged in turned on.
10. Who said you could download all this crap?
11. No thats not what i said.
12. Thats not the question you asked the first time.
13. Asking my boss won't speed me/the LAN up.
Was it supposed to be 10?
And just remember that if we ran the networks for ourselves only they would probably have more problems due to tinkering. But at least everyone would be walking round the office with cordless headphones, mic chatting with gtalk.
They fitted George Orwell's coffin with rollers so he could turn over more easily years ago.
One of the most frustrating things is users who do what you ask, and then promptly do a bunch of things immediately afterwards that you don't ask. You try going step-by-step with them, and meanwhile they are opening menus and clicking away at things they don't understand, because somehow hearing your voice tell them what to do gives them all the control of a runaway horse.
... for continuing to prevent /. from ever being a serious news source with his constant and endless supply of pointless and stupid articles.
1: Users can tell you exactly what is wrong and how to fix it.
1a:Don't listen to them.
1b:If for some reason they are right, make them do something else first and tell them that fixed it. Don't want them to become too cocky.
No matter where you go, there you are.
#1, "Users Lie"....great example: set up a workstation for a guy...fresh win 2K installation w/ all service packs, etc. later he complained about spyware being on it, and that he couldn't find some file amongst his documents. so while searching for the list item, i found porn in his network share...with him standing there watching. i just laughed, and being a nice guy, cleaned the spyware out for him. here's the kicker: he claimed the computer belonged to the president of the company before him, must've been him that got "that stuff" on there. in his network share. and he wasn't kidding.
using anti-bacterial hand soap is like drying your feet in the middle of a shower.
- You're like an offensive lineman on a football teeam; no one knows who you are until something goes wrong.
- In an emergency, shut the door, turn off the phone, and work the problem. If your manager isn't answering user queries, that manager is doing a bad job.
- "If you don't know what to do, why do anything?"
- Be prepared.
- ALWAYS update and refer to the log book.
- There's no shame in saying "I don't know;" pretending that you do is a sin.
- Keep a spare.
- No development on production equipment.
- You're there to fulfill the user's needs--not dictate them.
- Listen!
Now let me just kill that last background process with the old 'kill %1'
[$researchgroupserver]: kill 1 enter
Crap!
No matter what you do or how many hours you spend you will NEVER be appreciated.
Treat users with respect even if they are clearly in the wrong. Don't patronise somebody if they haven't got the first idea about computers: educate, don't insult. I'm not a buddhist but the old karma idea of "what goes around, comes around" seems to play out in the long term. Being patient with somebody who's royally screwed up their computer pays off in six months time when you need them to put your expenses claim through accounts at 5pm on a Friday evening/ notice you standing in the rain by your broken down car/..../
Sometimes users just break things from being absentminded or a little careless. Like the time I was called in to replace a floppy drive and realized the secretary hadn't removed the tiny plastic sleeve at the front edge of the disk, jamming it in the drive.
Users will lie to cover their asses like the article says, but if it's an honest mistake, I smile knowingly, point it out, and tell them not to worry about it. I'm contract, so the company gets billed, I get paid, and the client's staff get to avoid unecessary embarrasment.
%98 of statistics are made up on the spot
learn from yesterday, plan for tomorrow, party tonight
or one out of three ain't bad
I know, those are all corrolaries of Murphy's law, but hey.
[Pruneau
Ten rules? Not enough - we are dealing with lusers here. With lusers, you can guarantee but one thing - they will find new ways to completely fsck up a perfectly running system. IOW, "No one could be that stupid." is by _definition_ false.
Lusers are scared of the computer (Fear what you don't know), scared of what it does (Fear of learning) and scared of making it work properly (Fear of looking like a fool). What you need to do is provide more fear (Fear of being LARTed into oblivion by the sysadmin), which will negate the first 3.
Soko
"Depression is merely anger without enthusiasm." - Anonymous
Step 1: Reboot!
Step 2: if the reboot didn't work, format and reinstall!
That should be all the help documentation M$ gives you, besides the crap for help they have now
#11. Seed AI installed on your users' machines will cause a Technological Singularity in 2006.
#12. Hotshot AI coders will waste countless hours learning to program good old-fashioned artificial intelligence (GOFAI) in old-fashioned stack-based Forth.
#13 Association for Computing Machinery will use your organization as a poster-child test-case for Seed AI gone amok.
#14 AI4U will lie around on people's desktops as a mark of prestige and sophistication, or as a last-ditch Christmas gift for obnoxious know-it-alls who have never been truly challenged in their pitiful lives.
#15. User Manuals will be totally disregarded or rewritten and sold on eBay.
Even if you've been doing this for 20 years. If you are working with another technician, have the grace to treat them like an intelligent human being.
bun-fhuinneog agam!
Reboot, Reinstall, Upgrade
Been there, done that, paid for the T-shirt
and didn't get it
Whip me, beat me, and prove me wrong, but the use of trivial passwords for most user accounts can save a lot of headaches in many (special, small, non-DOD type) organizations. My observation and in practice has proved me right over the long term. My reasons, 1st most folk cannot remember their own passwords, even when they type them in every day, and with password managers they are not forced to. They also, usually write them and post them on their monitors. If they can remember a trivial password, it saves me having to reset it once a week. Further, if they can remember a password, then they can figure out to change it to a non-trivial one. If they are working on anything sensitive, they KNOW the importance of secure passwords and will use them. If the information is not sensitive, it is to boring to waste time on anyway. If some smart, evil type is in your organization, you will recognize them quickly anyway and will stand out like a super model at PTO meeting. And of course, never share root.
Please mod me 1 or troll. It's where the truth is these days, even on Slashdot. Beware the power of moderators everywh
Where I work, we have terminals mounted in mobile equipment for inventory tracking/locating. These terminals do not have a mouse, just a keyboard. The company has a network policy that pops up a plain message box when you log in explaining their network policy. Of course, since these terminals do not have mice, we have this policy notification disabled for the auto-login on these terminals (I believe the users just sign a piece of paper acknowledging they read it and agree to it. And yes, we do find it easier to have them sign a piece of paper then teach them how to use windows without a mouse. Once they log in they simply use a telnet-like program to access the inventory system.)
Anyway, one day I get a call that somone using one of these teriminals had to restart his and now there's a box on the screen about some "network policy." I grab our local sysadmin and inform him about it, and he comes back with "that's impossible. That's disabled on those accounts." Sure enough, I took remote access to the computer and right there was that darn message box. The sysadmin was speachless. Turns out someone in another branch, who obviously knows nothing of our setup, thought we were violating corporate policy and enabled the warning. You can imagine the administrative storm that ensued.
Just goes to show you. Even if you think it's impossible... someone, somewhere has made it possible.
(Posted anonymously for obvious reasons)
1) Apply this patch only if all criteria are met
2) Apply this firmware/bios update only if there is a specific need for it in your system.
3) The default kernel is fine for most applications.
4) Leave well enough alone.
5) It can wait until Monday.
It's only funny until someone gets hurt. Then, it's hilarious.
This is related to a wiki article I posted recenty at http://www.theinfobox.com/index.php/Portable_USB_A pps. It is sort of an open letter to end users from systems admins. Because its a wiki, you guys can can add your own tips from personal experience.
Even if there's no way it could possibly be causing the problem, it's nevertheless probably the cause. Can't run a particular program any more, just after you upgraded that old switch? Put the old one back. You'd be amazed at the level of interaction you get between parts of a computer system.
I am trolling
No One Ever Got Fired For Buying Microsoft.
Including Sharepoint?
Note that #1 is usually related to #2....
Tech support is a whole different ball of wax, and I am afraid many of you are confusing what the top 10 list was implying.
Sys admins are internel tech support, for users within an office. We are not talking about technical support in general, calling outside of the office.
The difference is that external tech support staff are truely idiots (sorry, its true), reading knowledge base screens you could access yourself online and doing very little to actually resolve your problems. These people are not highly educated, trained, or knowledgable about the products they are supporting. How else could a call center in India deal with a product made and sold entirely in North America? Most call center staff in India have neverever seen a computer outside of work. You would be better off discussing what spices to use in your favourite curry over how to get a Dell computer up and running after installing a corrupted Windows XP DLL driver. Same goes true if the Dell call center is in Texas, just talk about chili instead of curry.
Sys admins dealing with internal support calls have to deal with unruly and ignorant staff. They are actually skilled computer users trying to deal with computer lliterates that think they know more then the poor sys admins that are generally overworked and underpaid. Or even worse, sys admins trying to deal with know-it-at engineering geeks that can't believe they did anything that might cause their computer to stop working properly.
Anyways, the NUMBER 1 RULE for office staff is to TREAT YOUR SYS ADMIN like a GOD, and you will find your computer is ALWAYS working, ALWAYS up-to-date, and if you treat them really good, you might find yourself with a computer far better then even your boss's computer.
I haven't thought of anything clever to put here, but then again most of you haven't either.
If you remove Solitare from the system, the people will raise up and attempt to over throw you.
Click Click Bloody Click PANCAKES!
The Russians have won. They have made the world a cesspool of distrust, greed, fear and hate.
Vendors Lie - they will tell you thier solution will do just about anything. They will add features that they never tested or used themselves just to put it on the web. They will say a feature is coming in a month. They will claim to cure cancer and save you a ton of money. Always test the software yourself before buying solution. If they can't give you a demo copy or setup a test server in your shop for you to test real scenarios, look for another vendor.
But junior admins and new hires will always believe otherwise.
You realize that explaining the joke is a mortal sin, do you not? Ju kill de funnay. See you in hell!
I have a friend living the GeekSquad life. I'd never hire him as he believes in their process to fix lockups:
1. It must be this unsupported software: remove Firefox or any F/OSS.
2. It is a virus, your AV is no good, purchase Norton CoverYourAss v9.6 for $49.95.
3. The AV doesn't perform a deep clean by itself, we can run one for $24.95.
4. You need a bigger hard drive, w recommend Norton Ghost to copy it. $199.95 + $49.95.
5. We should install the drive. $24.95 + $8.95 wrist strap.
6. We should run ghost for you, $19.95.
7. You need USB 2.0 ports for your mouse to run faster, $49.95 plus $24.95 installation.
8. Your hard drive cables are old belt style, you needbthe snappy round cables, $29.95 plus $9.95 installation.
9. Your video board is old, the ATI MegaWow XL is only $199.95.
10. You should probably buy one of our Compaq BusinessPro by HP combinations, you burned your TCP/IP converter with static.
I pop open the discarded PC, replace the processor fan and blow out the case. All is fine - $30.
My truism:
The really big technical stumpers are often two or more problems at once. If you're nonplussed and scratching your head and your workarounds aren't working, make sure you're not just trying to determine a single root cause.
After an extended support session with a user...
Tech: Ok, we are going to test you system's head space and timing... Can you find the power or reset button or switch?
User: Yes?
Tech: Alright, I want you to rapidly press the button or flip the switch... Hears clicking in the background
User: Uh, Ok... Nothing is happening; the screen is flickering...
Tech: That's Ok, just keep it up... After a minute, there is a loud POP in the background
User: *Cough* There was a bright flash and now smoke is coming from my machine!
Tech: Hmmm, just as I suspected! Your system head space and timing was bad and you just had a MBS [Magic Blue Smoke] failure. Here is you RMA number you will need to pack the system up and ship it to our depot for diagnostics and maintenance... We will have to ship you a replacement. It will arrive in a week... Unless you would like to pay the shipping to recieve it overnight?
User: No, a week would be fine...
Really happened, I swear!
1) 90% of the root access and database passwords haven't been changed in the last 3 years 2) 90% of the files are chmodded rw-rw-rw-
Isn't it ironic that a web site about sys admin problems has given a system admin another problem by slashdotting it?
My current favourite question, when people's monitors don't come on after they've moved the computer, or got a new one, is "Is there more than one monitor port? Have you tried both?".
:)
They always claim there is only one socket the monitor will plug into, and without fail so far there has been an onboard one, which they are using, and one on a card, which is the one they should be using, and have completely missed
Combination - fun iPhone puzzling
Andrew's List of Project & Server Truths
1. End users lie. They don't mean to lie. Often, they don't know they've lied. But they lie. Don't believe anything they tell you.
2. If someone says "I didn't change anything" it is a lie. If nothing changed, it would still work. They simply don't know what they changed, or don't want to admit it.
3. A temporary solution for a few users is never temporary and never ends up being for a few users. If you let a temporary solution onto your server for those few users, you will be supporting it as production code forever.
4. There will always be a big plan in progress to replace what you're currently doing. 90% will fail, having served the sole purpose of delaying your funding for needed fixes and updates to the present (working) solution.
5. When the "Next Generation" project is complete, it will end up being much more similar to the existing generation than to the original "Big Idea" which lead to the "Next Generation" product. That is to say, all new projects end up leaning further and further toward incremental changes the longer they go on.
6. If you outsource your project work to low bid suppliers, you'll get exactly what you asked for. It will not, however, do what you thought it would do. A good supplier will instead work with you to understand what you want, not what you say you want.
7. Most project managers aren't. They're budget managers. The more technical ones think they're project managers and end up causing huge problems. If someone is really a good project manager, they're not holding down a a mid-level salary as a department manager in cube farm. The skills are too rare and valuable. It would be like accidentally finding diamonds mixed with the plastic beads at the dollar store.
8. Most hardware failures are not. They are configuration or software errors that someone couldn't figure out. Replacing the hardware worked because it forced a reinstall.
9. Sometimes, servers just crash. Its sad, but true. Call it cosmic rays interacting with the RAM chips (Which in theory, does happen from time to time). Start by just rebooting the damn thing and waiting for it to happen again. If it doesn't, problem solved. Don't take it offline for 24 hours to figure it out.
10. The most active data is the most likely to be destroyed. This is the same reason why your old cassette deck in your 1984 Chevette always ate your favorite tape -- because that's the one that was in the deck. The old tape you won't admit to have owned from "KC & The Sunshine Band" never got destroyed because it was in the back of the glove compartment hidden under the registration where the deck couldn't get at it.
-- Andrew
The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
"Rule #10 - The Holy Grail of Tech Support is the reboot"
If you believe this or if you need this, you are running a
POS operating system and its probably from Microsoft.
That this would even be considered a rule by a professional IT
worker is all the proof we need that Bill Gates has caused
more damage than he can ever hope to make up for.
What utter crap.
1) Document everything: what you fixed, how you fixed it, error messages, etc. It makes it so much easier to look at a list of common errors users in a particular company encounter. It is also great to say that you cleaned out spyware x # of times in the past month from one machine.
2) Patience is key: Explain questions in simple terms and a user will provide the information back. Might take a little work to understand what is exactly going on, but if the user loses confidence in you, then you won't have an easy time finding/fixing the problem.
3) If you can, be there in person (or at least remotely): If you can see an issue with your own eyes, the solution is likely to come very quickly. If you are forced to do it over the phone with no remote access to the machine, it is going to take a long time to solve.
On a 24x7x365 job, I learned the value of walking through the user's work area every weekday morning, first thing.
They started waiting for me to stroll in instead of paging me at night, just to be nice to me.
But the best part was, they thought of me as the guy who keeps the system running, because most of the time that I showed up, the system was running.
My colleagues who only showed up when their systems broke had the reputation "Here comes trouble!"
Taking credit for things going well is essential!
MICROSOFT is an acronym: My Intel Computer Runs On Software Often Failing Totally (c) 2005 an_unknown_soldier
"Learning is not compulsory... neither is survival."
--Dr.W.Edwards Deming
1. 'The internet' can help find someone else with the same problem, problems are not unique.
2. If it is unique, then you're screwed.
3. Slashdot decreases productivity, but increases conversation topics over lunch.
4. Never switch mail clients for your users, you will need to make the new program to behave exactly like the old one, which is impossible.
5. All Windows problems can be explained by making up words, dancing around, and waving your hands, users will accept that.
6. Backups that aren't scheduled never occur.
7. Make sure there are an odd number of people in any group decision making process
8. If I switched all the desktops over to linux, I'd have a job for a few months and then get fired, luckily they don't want to switch to linux
9. Never upgrade unless there is a problem.
10. The desktop with the most important information will be the one that is most likely to be infected, or have a hardware failure (Variation on Murphy's Laws)
__
"I'm not drunk, I'm just exhausted from drinking all night."
-Peter Griffen
... for the sake of all mankind, test your backups. A backup strategy creating backups you could never use to restore your data is no better than not backing up at all.
Just don't create a file called -rf.
10) Patch Current. Then ask for the unreleased patches. Then ask for development involvement.
9) Patching only works 30% of the time
8) Metalink is like a massive "Magic 8 Ball" that pulls responses from the database. Treat it as such.
7) Tars are the same as 8, except you have a customer service rep reading the 8 Ball.
6) If it generates core files it's the DBA's problem.
5) It's ALWAYS the DBA's fault.
4) RMAN is your friend.
3) You know more about Apache than Oracle does.
2) Oracle won't admit this.
1) Autconfig doesn't.
What if it is just turtles all the way down?
IT is neither.
Great. Glad you feel that way. Now, before we all hug, skip, and fling daisies, you need to remember ONE thing:
THERE ARE NO FEELINGS IN IT. EVER.
Feelings are reserved for secretaries named Gladys who come crying to you when they accidently delete all the pictures of their grand kids.
"This article is very comforting."
You better sack the hell up if you are going to make it in the IT world.
"I'm just here to regulate funkiness."
I was under the impression that a lot of errors were attributed to ID10Ts? ;-)
Dream as if you'll live forever.
Live as if you'll die tomorrow.
~Anonymous~
I like how he contradicts himself with #7 and #9. Use Linux over Microsoft and you will save money...NO...use Linux over Microsoft and you will be fired!
"#7 - No One Ever Got Fired For Buying Microsoft
So sad but so true. This old saying used to reference IBM, but oh how times have changed. Linux may be powerful, but the command prompt and configuration files and filesystem obscurity will just as soon get you a pink slip if something goes wrong and no one knows how to fix it but yourself. Even so, with as much stupid crap as we admins have to put up with on a daily basis, configuring some of the 'high end' Microsoft software is enough to drive you insane. Ever tried installing Exchange Server or, worse, installing Exchange Server and migrating a 5.5 install to Exchange 2000? I feel your pain, oh how I feel your pain."
"#9 - Know Your Needs
This one could also be called "Learn Linux." Many admins get wooed into the idea that "managed solutions" are always the correct ones. A web interface on a switch is cute, but rarely useful. A huge Cisco router may not always be necessary, sometimes a 'lo-fi' approach is best. When you want a spam solution, before looking at $5,000 servers and huge licensing fees for Windows Server software take a look at one of those old 'junk' PCs you have in the closet, download your favorite distro of Linux, and install procmail and spamassassin. You (and your budget) will thank me later."
-Joseph
seen it before. no digg.
...unless you've got a build process that ensures it or testing tools that compare them for you.
Backups are crucial? C'mon, real men don't backu+++ATH NO CARRIER
...is the result of trying to implement 100% of user requests. Sometimes, telling the user "no, you simply can't have that" is the best way to ensure an application isn't horribly poisoned by thousands of totally irrational, non-intuitive crap "features" each piece of which makes sense only to the person who requested it. Worse, such design-by-committee applications are invariably written interface-first, back-end last with no regard to how to actually make the goddamned thing WORK, much less work efficiently.
I agree, good software should be intuitive, but far better to be proactively engineered to be more intuitive, rather than reactively veneered to feel less unintuitive.
212% of statistics are made up
I joined a company recently which had a very bizarre problem-solving M.O.: they didn't use google. at all. ever. It wasn't that they used something else like Yahoo, they just didn't do the whole "check to see if anyone else has come across this before" thing. This was -extremely- strange from my perspective, I just couldn't understand it.
.NET, and some proprietary products and libraries for web portals and the like. I soon discovered that for many of the problems I encountered, googling was no use, because there just didn't seem to be any (useful) information out there - this was mainly because the code relied so heavily on closed (and very badly documented - or undocumented - APIs). You either figured things out by yourself using the old bang-head-against-wall-until-it-works technique, or you figured out another way to do it. It was an awful way to work, and one of the main reasons I left the company soon after joining them - I just couldn't work like that.
The company was in software development, using windows, java,
But even when we had problems with widely used and understood software, i seemed to be the only person who actually went looking for solutions 'outside' the company. I remember one problem with CVSNT which took me about 10 minutes to find a solution for on google (and even less to implement), but which had stumped all the devs at the company for months - and they were all impressed when I had it fixed within 30 minutes of saying "I'll look into it". a similar ethos pervaded the whole development team (to be fair a couple of them were much better at finding solutions elsewhere, but those guys, ironically, were no longer so 'hands on'). I know it's a cliche, but if it wasn't microsoft or java, they wouldn't even think of it - not through a blinkered outlook as such, but just because they had no concept of other possibilities existing. They were all good at their jobs - there were some very good developers there; but outside their usual 'sphere' they weren't that good at problem solving.
it still seems strange to me now. google is the best development/admin/debugging tool I have (before that it was hotbot, altavista - the names change but the use was always the same).
this leads me to a truism: it usually takes far longer to find the solution than to implement it. it is therefore often better to know how to find information than to actually 'know' stuff. but, this is more true for open-source software than for closed-source, as the former has exponentially more information easily accessible than the latter.
It's not in the Wayback Machine, but Google has cached the article.
So which one of these truths applies to this server being down?
Change there password to the entire text of the Da Vinci Code, except for the parts you don't beleive, and make the system lock itself after 2 seconds of idle time.
This sig is false.
New code is rarely better code.
If you break the internal webserver at 7.30 on a Friday, you've got all weekend to fix it before anyone else comes in and realizes you broke it!
--LWM
This is less of a rule and more of a guideline, but, people who have no idea what they are doing will usually somehow manage to do what you didn't think was possible if they have any power over the system at all.
"Quoting yourself is stupid." -Me
My version of this rule is slightly different.
"Users will always tell you what they think they should have done, not what they actually did."
Suppose you were an idiot. And suppose you were a member of congress. But then I repeat myself. -- Mark Twain
My standard pep talk:
Users are idiots. This is a good thing.
We expect them to be computer illiterate, and they rarely disappoint.
If I'm working at a biotech company, I don't want the researchers to be good at computers. If I'm working at an investment firm, I want the users to understand investments, not DLL's.
We're here precisely so that they can be idiots at computers... and experts at whatever it is they do when their computers aren't broken.
The company isn't here so that we have a network to play with.
Learn to praise the users' idiocy, they'll appreciate it.
If the users get frustrated, empathize with their confusion and blame Microsoft. Never fails.
Seriously, anthropomorphizing machines is a powerful technique. It gives you an approximate but effective mental model of a complex system. "Primitive" cultures are not dumb when they attribute personalities to objects. Our brains are wired to use personality to predict complex behaviour.
My Mother had no technical skills or knowlege - but she treated the automobile like a pet. She was alert to the tiniest change in sound or vibration of the machine, and very often alerted my Dad to problems long before he was aware of anything. One time, driving across country, my Mom said the right front wheel "didn't sound right". We were cruising along at 70, and everything seemed fine. But she insisted, so my Dad pulled over and checked all the tires. No sign of a problem. He pulled the hub cap off the right front wheel - and noticed that the cotter pin had broken! A few more miles and the wheel would have come off. My Dad panicked, since we didn't have any cotter pins in his repair kit. But my Mom dug in her purse and offered a bobby pin. My Dad didn't want to use it, because it was the wrong kind of metal and would break easily. My Mom said she had more, so he put it in. That bobby pin took us another 5000 miles.
My Dad does all his own work on his cars - at least he did until he ruined the valves on his Honda Accord a few years ago. Now he lets a mechanic do some stuff for him. I learned to be in tune with machines from my Mom, and learned to fix them from my Dad. When designing file system software back in the '70s, the rhythmic sounds of the disk access mechanism was my best feedback on its efficiency. Those were the days of 14" disk platters.
No One Ever Got Fired For Buying Microsoft.
No, but people have been canned for having major network issues, downtime, and other problems - oftimes in conjunction with migration to a particular MS product. There are admins that can make their corporations run well on MS technology, and some can't. Be sure which one you are first (the same applies in reference to migrating to non-MS products as well).
Maybe there is some truth, but people don't usually get fired when they are the only person who knows how something works......... *writes this as he submits a 1000 line badly coded file with no comments....*
In undeveloped countries, the consumer controls the market. In capitalist America, the market controls you.
On auto-sensing switches always lock ports to 100 MBps / full-duplex operation. Auto-negotiation just doesn't work reliably in my experience - monitor switchports running auto-negotiate and you'll see many errors especially when the interface has some load on it.
More server centric.
- dont install non-essential software, especially on a server.
- Never unrack the unknown server because it "looks unused". Disable/unplug its network port and label the server when you did it, and leave your number. Unrack after 2 weeks.
- The serial console is your #1 friend (if unix). The recovery console is your #1 friend (if windows).
- buy something that supports remote power cycle, or buy an external device like an APC masterswitch.
- monitoring is your friend. Centralized syslog servers can do a lot of correlation. As can snmp trapping and querying. "see the graph".
- Accurate time is essential. Dont forget to update the ntp time to the hardware clock.
- Dont change too many things at a time. Rolling code, upgrading kernel and applying patches (or service packs) are good things to do one at a time, but usually a bad idea in a production environment. The next morning you get "the network is slow" or "performance is bad". Is it the app, the kernel, the updated network/san/scsi drivers? Good luck.
- your problem is probably not unique. Check google. Check the MS knowledgebase (if applicable)
- gather data before acting. Is this a problem to one server, many servers? Does it match a change in the environment? Also known as the "Why is this box running slow all of a sudden? Let me reboot it. Wait now this box is running slow. Oh I'm getting DoS'd from one ip. sorry about that reboot."
- TEST YOUR BACKUPS. - Luckily these days its cheap to do disk based backups so this should be simple. Offsite backups are an entriley different issue though.
Proof right here: http://www.thinkgeek.com/pcmods/cables/6b35/
Nah, carbon based errors, instead of silicon based errors. 99% of people won't get it - unless you work at a chemical plant!
There are two types of people in the world: Those who crave closure
Many reported network problems are coming from another source. User A's computer locks up the network, and A ignores it, user B reports thier computer is locked up. User A restarts computer in frustration and every thing is fine again until the next time. Techie never knew about user A's problem.
Don't rule out Word or Excel as not being a cause of any problems.
USB device problem? Might be a problem with your printer, had one where the keyboard and mouse were seemingly dead, it was the printer locking up the bus communication.
"Enjoy what you're doing! If it becomes drudgery, you're doing it wrong!" - Jim Butterfield
1) Document everything. I've had coworkers who thought being asked to document their processes and procedures meant they would soon be canned. If you document your processes, you can pass them off to other team members when you tire of them, so you can move on to bigger and better problems. 2) Talk out loud when working with users. It was a habit I got into while doing field service. Explain what you are doing while you are doing it and a) the users may learn something, and b) it lets them know you're not related to Nick Burns (SNL). By taking the time to explain things (knowing most users won't understand any of it to begin with), the users will know that you are interested (okay, some may feign interest) in their problems and the resolution. After doing this for years, I have seen many technophobic users start to come around to where they will actually try to fix a problem themselves before calling the help desk. 3) Problem always happen on Fridays just before quitting time.
I use irony whenever I can, but my shirts are still wrinkled...
Reaching way way back here. Had to fix an old PC clone from Olivetti. Symptoms pointed to heat problem - ran fine for a while, then random glitches and lock up. Took off cover, vacuumed some dust off the motherboard. Didn't look like enough to cause a heat problem. Sure enough, keeps on locking up as before.
Then I notice that the case (desktop, not tower) has vents all along the very bottom edge. Pull off cover and then pull off motherboard. Well great thunderin lard jaysus, there was a layer of dust _under_ the motherboard a half inch thick and dense as felt. It peeled off in a single solid rectangle you could have cut boot insoles out of. Once the dust mat had been removed and airflow thru the case restored it ran fine.
None of them can see the clouds; The polished wings don't care.
I am a Microsoft Certified Reboot Specialist.
Web 2.0 == Giant Blogspam Circle Jerk
This list is for on-site support techs, not Systems Administrators. A real SysAdmin would *never* reboot a Production system unless it was absolutely required (or running Windows). If this were a list made by a real SysAdmin it would read a bit more like this:
#1. The User has no idea what you do, but they will blame their problems on you.
Some guy on the support desk bumped a call to you saying you had a server problem, the user doesn't have internet access, forward it back to the desk.
#2. Other employees have no idea what you do, and will try to pass the buck to you.
Exchange is down and you're a Network Admin? Well suddenly there's a network problem, certainly not a problem with the Exchange server. There's a network problem? Well then you're a Windows Admin and it's clearly your fault. No matter what they will always find a way to blame their problems on you.
#3. If you're doing your job well, they will fire you.
Congratulations, your systems stay up all the time, maintain themselves, and building a new system for your environment is such a painless and well-documented procedure it takes minutes and a monkey could do it. So we're hiring a monkey. Get out.
#4. If you're doing your job poorly, they will promote you.
#5. A crashed system holds many secrets, do not reboot it.
You need to find the source of the problem, or it will come back to haunt you. If you're fine with rebooting six times a day, be my guest.
#6. Backups are important, but multi-layered redundancy is the way to go.
Why have one webserver when you can have two? Why have one mailserver when you can have two? Have two locations? Why not have four mailservers? The more redundancy you have in your systems and your network, the less you have to care about midnight outages.
#7. If someone needs to tell you to be more polite, you have no business working in IT.
Seriously, you screwing up screws up everyone's day. You need to be meek, friendly, and try your best not to let everyone know you make more money than them. Except Sales and Marketing. They scoff at your puny salary.
#8. Always ask for a lot more than you need.
A big project require four servers? Ask for eight. Setting this up is going to take a week? Tell them it will take two. It never hurts to try to get a safe buffer, and you'll never get what you ask for anyway unless you're amazingly lucky. Either way, if you say it takes two weeks and you get it done in one, you look awesome. If you tell them it takes eight servers and they give you four, you look like a rock-star when you get it to work. If they give you the eight servers anyway...well...it looks like you have some spares for once.
#9. If it can't be done, say so.
Don't get yourself involved in an impossible project. Doing the impossible might be part of your job, but if you don't know OpenView and your manager wants OVO to be making his coffee in the morning, don't tell him you can make it happen.
#10. Always look for something to improve.
This is what most admins forget. An idle admin is a fired admin, and an idle admin eventually is a stale and clueless admin. Remember, your manager will never hate you for suggesting new projects, and for suggesting things can be done better, especially if it's free. I find when I'm sitting idle between projects there's nothing better than to give myself a project and FINISH it. There's nothing quite as nice as informing your manager that there's a better way to do something, and that you've already set it up.
http://www.computerworld.com/news/2000/story/0,112 80,51168,00.html
"A good idea is no match for a bad habit."
"Ninety percent of a system's cost is still training people to use it."
"Old ideas got that way because they proved useful."
"Data isn't information. Information isn't knowledge. Knowledge isn't manageable."
"Systems aren't made from metaphors, paradigms and methodologies. They're made from code, wires and hardware."
First question out of my mouth: Did you reboot? Also, anouther one of my favorites is when a user gets one blue screen or error and it never reoccurs, if it happens once it's a fluke, if it keeps happening it's a problem.
...is that NOTHING absolutely nothing can be hidden. If you roll something out with a problem in it, or something missing, they WILL find it.
:-(
Everytime I rolled out a config with some small error or missing component or feature someone found it and complained. I thought it was just coincidence or takig too long to path. Noppers, they have fingers and will click anything with a shorcut or [OK] button.
The latest gotcha for our dept. was the user who discovered that ".jpe" files were not opening. Two days of my life I will never get back
Heybiff
Even the Sun goes down.
Don't forget the often neglected 8th OSI Layer. The Carbon Layer, where 90% of the errors occur.
When modding "Informative", please make sure it both has a source and IS actually informative.
Truth #11: When something goes wrong, even the most low level intern is suddenly your boss.
Per ardua ad astra.
There is a variant to that:
EBCAK
Error Between Chair And Keyboard
Generally this is reserved for the PEBCAK which is a repeat PEBCAK for the exact same issue time and again. Unfortunately such cases are unrecoverable errors.
The Christian Right is Neither (Christian nor right). See: Matthew 23, Matthew 25, Ezekiel 16:48-50
Mess with it until it stops being broken and then leve it the fuck alone .
In two companies with 20 employees each, the one with the most IT problems will be the one with an IT guy on the staff. Because he (it's always a he in these cases) can't resist trying to upgrade or install or develop something which doesn't need changing.
The doctor gave him overdose...
Wait, so something goes wrong and they fire the only person who can fix it?
Sounds like a Dilbert comic.
This is the only "Truth" I really disagree with.
I have to say that if your having to reboot, and you don't know why you're rebooting, you don't know the system your working with well enough. Your essentially throwing a band aid at the situation. Find out the cause of the problem first and solve it. RARELY should anyone have to ever reboot unless your using really flakey software in which there is no software solution to fix it.
In a corporate environment, telling users to reboot first is probably the worst thing you can do in terms of solving the problem! Get on that pc, find out whats running, identify the cause and solve it- restart the process if neccessary. If you're able to at least isolate it to the process your half way there already.
Without finding the root cause of each problem, you may solve the surface problem right away by reboot, but rest assured the root cause of it (hardware/software) will remain unsolved and future problems will only snowball.
come from a connection that is looser than it should be
or a luser more connected than he should be.
-Lasse
1. Always ask for more than you need. More money, more capabilities, more features. You may not ever use them but it doesn't hurt to have them. Every graybeard alive has uttered the phrase "Why do we need a whole gig? A 500 meg drive is fine" and later regretted it.
2. Every call is a desire for a personal visit. Everyone who calls really wants you to come by and fix their problem. Some people like to show their coworkers that they have their own tech guy. Some like the feeling that they are important enough to get a personal visit while the other poor slobs have to settle for a phone stroke. Some simply can't find the esc key. They always want you to come by. My neighbors even call wanting a visit and they live up the mountain. Offer me Scotch if you want a personal tech visit.
3. Our computers at home have the covers off and we have a closet full of ISA boards. Admit it. You can tell us. We understand.
4. We get really pissed off when people call and say "I can't explain it. Just come by". It's ALWAYS something that can be figured out over the phone. Or they are trying to install illegal software. I can say "No" over the phone just as easily.
5. We have all given out passwords. "Come up with a password". "I can't" "Just type something in and write it down" "OK FINE...777777" "No you can't use that" "I don't know what to use" "OK, GaxtaL1967" "That doesn't make sense" "Exactly". And we've all dealt with people who forget the password between typing it in and the confirmation.
6. An illiterate computer neophyte can defeat any system we design to stop him from damaging stuff. He will zap his hard drive despite having NO rights and will have defeated the backup system months prior. He will yank down his password-protected firewall in order to download stupid videos and, as a result, try to infect the whole network. He will send a Word document to the entire company with a virus embedded in it because he somehow murdered his anti-virus software. And he will be an expert in computers who never fails to offer his advice to you.
If you aren't part of the solution, there is good money to be made prolonging the problem
I went through three power supplies before I discovered the fact that I actually had a power cable that was going bad.
I used to work for a company that developed a very highly customized package for our customers, put it on the *NIX of their choice, and installed it in their data centers. Although based in the US, one customer, whose site I was working on, was in Basingstoke, England.
The client was (and probably still is) a hard-core Big Blue shop, so the *NIX of choice was AIX, running on a two-piece RS6K machine. One piece was the server itself, and the other piece was an 8-disc SSA drive tower.
The drive tower had three power supplies, allegedly for redundancy, but these, in turn, were connected together via a three-way IEC Y cable. This then plugged into a normal IEC cable that then had the monster 13A plug they use in the UK on the other end. (If you haven't seen one of these, they're huge. If we used these in the US, we'd probably rate them for 50A).
The plug had a fuse in it.
I'll say that again, because this is important, but not something that you typically see outside the UK: The plug had a fuse in it.
After we hardware guys left the customer site, and left it in the capable hands of our software guys, we got a frantic call from the software guys that the discs had "just disappeared from the system".
To make a long story short (if it's not too late for that), the fuse in the plug had blown, thus killing power to all three power supplies, in turn killing power to the discs. Once we figured that out, we had our software guys get the customer's IT guy on the phone, he got out two more IEC to 13A cords and a fuse, and the problem was fixed in ten minutes plus reboot time. The Y cable was relegated to the scrap heap.
www.wavefront-av.com
I was going to say this was duped.. but then saw no other comments about that - I'm guessing I saw this on digg a WHILE ago.
The email is down.
Storm clouds are gathering and
You're a lightning rod.
Proverbs 21:19
It would be one thing if users just lied, but occasionally, you get someone really screwy. There's this woman where I work... let's call her 'Julie'. Julie is in the habit of totally fucking her Mac up royally. But when I ask her some simple questions, you know, "What were you doing when the screen went blank?", "Did you spill anything on the computer before it started smoking?", etc, all I get are denials. "I swear, it was smoking when I came in," or "I never just press the power button to turn the computer off." I have several witnesses who swear that she does just this very thing, every day. She'll go to such lengths (elaborate lies) to prove that what she says is true, that I can only conclude one thing: she actually believes what she's saying. Julie is crazy.
Anyhow, she came to me one day saying that her mouse was "jumping around like crazy". "Like crazy?" I thought. "I know someone like that..." So after, er, brushing her off a few times (I mean, what are the chances this is a real problem?), I decided to walk down to her office to see what she meant. Indeed, every now and then, the mouse cursor would leap wildly across the screen for no reason. After plugging and unplugging, and even testing all the peripherals on a different machine, I discovered two constants in the equation: wireless mouse and big ass monitor. A quick search on Google revealed that interference between wireless mice and other things, like monitors, is pretty common.
To wrap it up, I lied and claimed that I couldn't reproduce the problem. It was a real BOFH moment, I know, but it barely makes up for all the shit Julie's pulled over the years.
Users will most likely download useless applications (ex. apps that check weather, screensavers) no matter how many times they are told not to. I mean from where they get this you would think they could tell this stuff screams adware. Then when ask did you download anything on you computer before it started to have this problem, the answer is always no. Until of course you have to personally go down and fix the computer and they have to explain why this program was installed.
Note to self never mention Microsoft when posting on Slashdot!
... for buying Microsoft.
That one fucked up the whole list.
I'll remember to use switches that have started to fail though... that won't get me fired.
wait a second, I'm a software engineer... making fun of admins never got me fired.
-pyrrho
You want to spot the real pro in the machine room he (or she) has a vacuum cleaner in one hand and a Brother p-touch in the other. I honestly beleive there is a direct linear relationship between the efficiency and uptime of a shop and how anal they are about labelling stuff. I want to open a front door of a rack and see every server's hostname and every removable media device clearly labelled. I know YOU know that that CD-ROM drive is drive D: on the frodo server but I ain't got time to try to figure that out. Even more importantly I similarly want EVERY cable in the back of the rack to have some kind of useful label on both ends (unless it is less than a foot long then just one end is OK).
>I hope we all learned something.
I got to the end of the article and the tech in me read
"I hope we all killed something today."
"Love is like pi - natural, irrational, and very important." (Lisa Hoffman)
11.) Can't train 'em, can't shoot 'em.
I'm very good at what I do, not even 5% of my peers are as good as I am (admittedly I work on the helpdesk so the bar isn't necessarily too high in some cases). I know my stuff in a lot of detail (I'm a geek) and am usually the most intelligent person in any room I'm in. These are plain simple facts and even my employer wouldn't deny them, I am however (despite the seeming arrogance of the preceding statements) willing to learn and depressingly aware that I don't know everything (I generally find the more I learn the more I realise I don't know). I treat users as human beings and enjoy the problem solving parts of my job. Ok, so repairing an oversized .pst for the nth time is less than fun but I usually get all the difficult stuff no-one else knows what to do with. Fortunately my employer recognises this and my pay slip is suitably well padded. Getting someone with my level of knowledge who actually enjoys helpdesk work is worth the extra shekels to them, it means the systems and comms teams can get on with taking things forward while I make sure the current setup keeps ticking over.
Most users are perfectly capable of firing up a command line and following instructions if they're given clearly and unambiguously. Obviously you want to keep it simple (ipconfig, set etc) but it's the quickest way to get their IP address (assuming you don't have central login histories built in to your call logging software or it's not working).
This one makes me shudder. Repairing the damage done by those who went before me and rebuilding the permission structures ("user in the global, global in the local", it's not rocket science for crying out loud!) once the directory structure is sane (and incidentally only allowing list access to the root file share) has eaten up more of my time than I want to even think about.
And don't forget that accurate backup reporting is just as critical. Finding out the backup has failed the last 2 weeks and the software didn't report it is not something you ever want to go through (fortunately we also do manual checks). This is a sore point with me, one of those head->wall things I don't want to talk about.
This is the core of my job. I have to balance network integrity and security with user needs, frequently the "obvious" (to the user) solution is not acceptable in some way or other (wireless for example is an absolute no go area on our network) so I have to work out one that is. I'm here to enable users to achieve their tasks and goals, not to get in the way.
See above, it just doesn't happen on anything connected to the core network.
no doubt.
-pyrrho
Hanging your certs on your cube wall is soooo 1995. Please don't do it, you are just embarrassing yourself!
I can't stress enough how valuable one of these, or some other good LiveCD, can be. If the box is Windows, Linux, whatever, keep one handy. One of these things can be priceless if the thing refuses to boot properly, someone deleted NTLDR, X locks up on runlevel 5, your driver interrupts conflict, a recursive script uses all of the PIDs, or any number of problems. Keep a printout of the boot options for the disk, too, to boot the unbootable.
I am probably by myself, but when I saw this I expected to see funny anecdotes about cages, colos, raid arrays, and redundancy. Not how to fix a printer and cleaning desktops.
...wait at least 24 hours to read an email. Do not reply to any email until the sender has sent a follow-up email. Do not do any work until the requester asks his boss to talk to your boss about why you didn't respond to the service request.
Your help desk staff are your best friends, as it's their job to keep the idiots with their stupid questions away from you.
Conversely, everyone else's help desk staff are a bunch of morons whose job is to keep you from getting a straight answer to an important question.
I reckon this has most to do with approach... users, especially the non-techy variety, tend to approach things in the same casual way they approach TV, or writing a reprort... casually, and intuitively... we aren't like that, generally; geeks are methodical... every step we take is scripted, and we're analyzing what we're doing as we're doing it...
remember trying to get those first couple of computers to talk to each other when you were a kid? one of the things we learned from that was approach: mentally cataloging each step along the way so it could be duplicated later... we deal with most things (and *especially* troubleshooting things) with the scientific method *firmly* implanted at the front of our conciousness
when the dvd doesn't work right for the non-tech, it's probably error related, but they wouldn't know that, because they just did what *feels* right... our "aura" is our ability to approach things methodically
It seems getting Farked and /.'d is bad for you bandwidth limits.
Bandwidth Limit Exceeded
The server is temporarily unable to service your request due to the site owner reaching his/her bandwidth limit. Please try again later.
Good thing I already read the article and was just browsing his site.
Not really for administrators, but...
When you'd bet your life that your code is right and the problem is in the compiler, 99.9% chances are the fault is yours.
Picard: Everyone dies alone.
That's a Mal Reynolds quote from Firefly, not a Jean Luc Picard quote from Star Trek.
(Of course, just because of the fact that I knew that, it's a safe bet that I will die alone too.)
Information wants to be anthropomorphized.
Bandwidth Limit Exceeded
The server is temporarily unable to service your request due to the site owner reaching his/her bandwidth limit. Please try again later.
Apache/1.3.33 Server at www.misterorange.com Port 80
Dude, I think that connector is loose..
I've personally seen this happen all the time. Someone tells me "this doesn't work" and the moment I type the same command or push in the PCMCIA card myself or whatever, it suddenly works. We dubbed it the Proximity of Genius Effect and is similar to the following koan:
2-9 are generally just variations of #1.
I had one today: "Someone [in a different company] called and says she can't send us email. She got a message that said it can't talk to the server." He's trying to be helpful, I know, but really, he should know that I don't know what "it" is, and (b), I highly doubt that's what the error said, unless it was an Outlook error, in which cases, what the hell does he want *me* to do about it?
Use Windows and find your own tech support!
Don't hang the hub by the cables, no matter how thick the ethernet and tiny and tiny the hub. Don't let 2m of cable hang from the switch on the high shelf down to the hole by the floor. It will work the first month or two, then will start to mysteriously fail. The most basic reason behind all these ports failing is that the cable puts stress on a port. Just attach all the cables half a meter away from the hub with ducttape or nails or staples or whatever, don't let them hang though, keep them loose, always leave at least minimal slack.
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
"Linux may be powerful, but the command prompt and configuration files and filesystem obscurity will just as soon get you a pink slip if something goes wrong and no one knows how to fix it but yourself."
Contrast that with:"# 9 - Know Your Needs:
"This one could also be called 'Learn Linux.'...When you want a spam solution, before looking at $5,000 servers and huge licensing fees for Windows Server software take a look at one of those old 'junk' PCs you have in the closet, download your favorite distro of Linux, and install procmail and spamassassin. You (and your budget) will thank me later."
Ok...., so which is it?
"Anyone that has ever gotten an idea based on any of my work and done something better with it-good for you."--J.Carmack
>>I know this was said as a joke, but I see this a lot amongst the geek community, the attitude that users just don't know what they are doing, and that is why they can't make anything work.
The reason you see it so often is because it is so true. User is a broad term, and I use it as anyone who is touching the system aside from the administrators (bosses, subordinates, that monkey down the hall). I work as a sys admin in a lab at the university I am currently attending and the worse things happen because a user thinks they know what they are doing when they don't. Here are a few of my favorite examples.
In the defense of the user, those who "know" what they are doing often makes mistakes as well. My favorite example of this is when the IT office calls and tells me that I have a virus on my webserver and it opened a port that is getting a large amount of traffic. I was told to fix the problem immediately or my port would be shut off. Naturally I asked which port... the answer? Port 80. After a short technical explaination they agreed to leave my port alone if I would at least do a scan. I scanned with a few antivirus and spyware programs as well as looked at a hijackthis log. Surprise surprise, the system was clean
'kay, some AC copy that to get modded up, I'd have to wait 20 mins now to post as AC.8 d9cfeacb1084ea2/index.html
http://www.mirrordot.org/stories/66f8b01be1874f07
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
When I went through Navy electronic technician school they taught us half-splitting as a formal procedure for isolating faults. Off the record, the instructors use to tell us to check interconnecting wiring first- 'cause it was usually the cause
:( the site ran out of bandwidth
I normally find the problem is located between a chair and a keyboard......
;-)
and sometimes it's my chair and keyboard.. odd that
When buying important servers, buy them with redundant power supply.
Really. This applies to important networking components as well.
Hot-swappable RAID really doesn't help when you have a single point of failure in the power supply. I've experienced several power supply deaths over the years and they have always been major headaches. I've also prevented one major headache by buying a double powered system.
Layer 0 - The JFDI's
Train the users to deal with a broken OS rather than train the company to ship a working one.
Clear, Dark Skies
I'm unlikely to report them to HR but I do have a standard "We know, please stop it" email that usually puts an end to it without any more official action.
"I'm using firefox+adblock and antivir just in case."
"You're ok then. Share cooler finds."
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
In those days, youngsters, we used ascii terminals rather like a single 'xterm'. Few people knew that these devices could 'scroll back'. This turned out to be the biggest single tool I used. When a user would say "It said this, and I did that", I'd scroll back, or check a log file, or do something else if I could. If what they said was inconsistent with these mechanical reports, I counted it as verifiably wrong. Otherwise, I just assumed it was probably OK.
I concluded that 40% of what users said about computer behavior in those three days was verifiably wrong. Sometimes it was wrong in a little way. Sometimes it was really, really, wrong.
I've no idea if this is still true: we're all a lot more used to computers and their ways. At the time, though, I realized that this is probably how a lot of magic tricks work: people think they know a lot more about what they saw than they really do.
I should emphasize that I don't think anyone was lying or really even that careless.
The Sysadmin truths I took away were: "Don't trust the details of user reports" and "Don't be mad at users for giving you the wrong information".
%cd ~/someoldcrap
%ls
[all crap. Checked.]
%rm *
somecrap: permission denied.
%su -
Password:
#rm *^H^H^H^Hpwd
/
whoops?
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
The power cable worked when you got it, then after a while it stopped working. Right before it stopped working, it "went" bad.
We're not talking about servers here... for all MS's flaws, most of their server systems are pretty immune to needing a reboot anymore... espescially if they have more than a single processor - Even explorer can be restarted without problems.
On desktops though, the reboot (although not nearly as common - I'd never argue that) is still a cheap and easy way to go. Next time X starts acting up, you spend half an hour playing with it, and I'll spend 5 minutes rebooting the box and getting on with life.
Haven't you seen My Name is Earl? Thanks to Carson Daly and your reincarnation as a Systems Admin, electronic devices bow to your strong positive karma and start working out of respect for the Dharma.
Karma. Oh yeah. Karma.
If I can't fix it, it ain't broke!
Subject says it all, how hard can it be?
This article was Farked over a week ago. http://forums.fark.com/cgi/fark/comments.pl?IDLink =1792019
#10 - Ask questions. The user might be able to tell you that the fan was really noisy but finally quieted down yesterday. Your friend might be able to tell you why your kernel won't compile. The guy in the shop might suggest that you replace a $2 wear part while you've got the thing scattered all over the room anyways.
#9 - Listen. Don't argue that the forbulater is good...just check it. The guy you asked is taking care of #8. You'll feel pretty stupid if you refuse to check it and it turns out the frob was fine but not properly seated.
#8 - Eliminate possibilities. If a sniffer shows that the ping's going out, you've just eliminated a huge raft of potential causes.
#7 - Fix it permanently. If the problem was hard to diagnose or it is easy to do, take action to avoid getting bitten again (like plugging that dead port).
#6 - Look for clues. I mean that literally: scorch marks, kinked wires, shiny spots and shifty eyes can all lead you straight to the problem.
#5 - Wiggle it.
#4 - Google it.
#3 - Have the right tools.
#2 - Trust your instincts.
#1 - If it worked perfectly yesterday, something's changed.
Oh yes, I agree to that strongly.
Sometimes they lie because they just don't trust the administrator.
During my last job, I learnt the skill of gaining users' trust from my supervisor. That is, sometimes you have to tell the user to do a bit of meaningless / distracting / crazy thing to make the whole troubleshooting process persuasive for some users.
(Posted as AC since I don't want my previous fellows to figure out who the user is)
For those of you who had touch your users' keyboard, you would surely saw a lot of dirty keyboards.
Yes, most keyboards are dusty if your company does not have a specfic person/group who clean computer devices, but what I mean is more than that.
In one of my previous jobs, there was a dirty (dark) and sticky keyboard on one of the user's desk. We cleaned it, replaced it with a clean one, yet it becomes dark and sticky again way faster than others'. We couldn't figure out why, until I saw the user eating a kind of snacks with bare hand while typing... and the nasty thing is, THE USER EATS THE SNACKS WHILE HE/SHE IS TYPING!!!
Couldn't figure out why he/she is not sick...
All maintenance activities are run from this management server.
2. Never use the user's keyboard -- see #1 and multiply by 10.
3. The user's keyboard usually contains items which did not stay in the user's mouth. Bring your own keyboard.
4. If hygenic input devices are not available, create a reason to work on the tower in your space where hygenic input devices are available.
5. If the system cannot be removed, engage the user. Make the user root/administrator thru remote interfaces; then direct the user thru the steps to correct the problem.
6. If you must use the user's input devices, maintain a supply of surgical scrub solutions in your personal toolkit.
Every mans' island needs an ocean; choose your ocean carefully.
Another networking tip is to use use the type of cables that have little hooks on the ends. Much better than trying to tape them or wedge them with toothpicks.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
The downside is that you'll have be more creative when making the blood sacrifice required to get the thing to work.
The upside is that fan blade cuts heal much more quickly than the flesh rips you get from stamped metal cutouts.
Dewey, what part of this looks like authorities should be involved?
Rebooting can cure ailments of all sorts...
I call BS on this one. Rebooting cures nothing. It merely covers it up until next time.
Most of what it covers up are:
Memory Leaks -- software that is incorrectly written, or uses libraries that are incorrectly implimented.
Disk issues -- if a disk drive is failing, replace it -- don't let fsck (or windows equivalent) keep it limping along - fix the problem because you will lose data if you don't.
While this may 'work' from an 'I need to get this guy off my back' standpoint, it fixes nothing.
Lodragan Draoidh
The more you explain it, the more I don't understand it. - Mark Twain
This might seem like an obvious one... but it's not. When a user complained about the password complexity requirements, when was the last time you told them about dictionary attacks? When a user complained that Internet Explorer told him the 'server' was down, did you explain what a server means in computer terms, or did you just send them off with a reassurance and a condescending pat?
Users are not stupid... they are ignorant. They don't understand why it is failing. They may even be very knowledgable, just not in the domain of the current problem.
While you're waiting for that reboot, why not explain to the user what you suspect the problem is, and why. When they get confused between their email address and their username, clarify and define the terms. When they put www in front of every URL, whether it should be there or not, explain about how hostnames are a custom, not a rule.
"Type email.example.com in the address bar at the top."
"It says host not found?"
"Read me the address bar, letter by letter..."
"http://www..."
"Hold right there... the address I gave was email.example.com. Not all websites begin with www, just most."
"Ahh, gotit, lemme retype it..."
"Hostnames are just names... we could have called it fluffy.example.com if we wanted, but that would be silly. *chuckle* Ok, now that you have that typed in..."
And hopefully that user will remember from then on that websites don't have to begin with www. They may even look at and notice the alternate hosts various sites use. They learned something, it took only a few seconds longer, and the user will hopefully know a little more about the background behind the stuff they are told to do.
If you take a few seconds out of every call to combat ignorance, pretty soon you'll start getting fewer calls. At the very least, the calls will be more tolerable because the user won't be making the same completely stupid mistakes over and over because they don't understand.
If every tech took a few seconds to combat ignorance, we could actually make a difference.
The Raven
"I will trust Google to 'do no evil' until the founders no longer run it." Hello Alphabet.
"System Administrator" is only part of your job title when something breaks.
cyn, free software and *nix operating systems enthusiast.
User: (some question you don't have time for)
Me: No hablo ingles.
(note - this works better during the dotcom-boom era when it was impossible to be fired)
Perfectly Normal Industries
'Course, it's not only from Firefly.
Dr. Lilian Thurman: What did Roberta Sparrow say to you?
Donnie: She said "Every living creature on earth dies alone".
Laws do not persuade just because they threaten. --Seneca
Hey, thanks for your comment. It was useful! Just wanted to tell you I got a good laugh out of this:
It peeled off in a single solid rectangle you could have cut boot insoles out of.