Life in the Trenches: a Sysadmin Speaks
Anonymous Coward writes "A senior systems administrator at a big ISP in Australia offers
a no-nonsense view about his line of work, the pros and the cons, ths ups and the downs."
← Back to Stories (view on slashdot.org)
suck my big FP bitch!
Pros: Cheese Doodles
Cons: Users
then be a system.... As heard on simpsons: "I maybe a hobo, but you are a nobo..."
Consensus is good, but informed dictatorship is better
getting dingoes in your servers.
...is Australian for beer. Actually, I've also heard that the Fosters in Australia isn't very good. But anyway, back on topic here... Someone needs to ask this guy... is there an Australian equivalent to, say, Water Joe or Bawls, for those late night hacking sessions? ;)
"To confine our attention to terrestrial matters would be to limit the human spirit." -Stephen Hawking
I didn't see the "problem solving skills" as a requirement for being a sys admin, mentioned anywhere in the article.
I think problem solving skill are a must for the sys admin job, especially if you don't want to be a Jr. Sys Admin and perform backups all your life.
I worked for a relatively large institution, in the capacity of a Sys Admin, and I know for a fact that you need some serious problem solving skills.
Consensus is good, but informed dictatorship is better
Good read - I think its important to recieve an impression on what your future jobs might turn into once you have been on the line for a couple of years.
Of course, its important to try your dreamsjobs during during university, but you never know if your dream wont turn into a nightmare after a few years but just working a few weeks there...
+++ MELON MELON MELON +++ Out of Cheese Error +++ redo from start +++
Dingo sticks in YOU!
Oh good God, I expect this kind of over dramatization from the popular press, but if /. starts it to.... "a sysadmin speaks"? wtf????
"The saddest words of mice and men, are not those which were, but should have been."
..."I'm sorry, does my smell offend?"
This guy basically has the "bastard operator from hell" mentality, he's just a little more polite about it.
Any sysadmin that has to log into a system while on holiday in *India* is a bad one. If you don't have enough redundancy built into your system that your junior admins/engineers can't hold down the fort for a week or two, something is wrong.
Second, "strong experienced based opinions" is crap. Open your eyes to new concepts and ideas. Like me trying to explain to two 10+ year network engineers that having a flat, layer 2 network across an entire Air Force base with 8000 users is a Bad Idea, and that adding layer 3 switching capability at the distribution points wouldn't slow down the network, and it would, in fact, be faster. Sure, hold on to your opinions, but understand things change, and if you don't change with them, you're a gorram dinosaur.
he's a sys admin, not a coder.
RTFA
What qualities do you rate as essential for a good sysadmin?
In rough order of importance:
Aptitude.
...
Is it just me, or is that a somewhat circular choice for first on the list? What IS aptitude, but the qualities essential for the purpose?
slashdot sucks !
booyaah !
... and he's given a whole new block of vacation months with the new year
you just knead to know what banner ADs to run MoSt. right? or wrong.
Having been in the IT field for 10 years, of which I've been a UNIX sysadmin for about 5, I must say this is one of the better articles/interviews I've read on the subject (not that I've seen that many). Not to over emphasize the importance of the job, or to inflate my own ego, but in all honesty I believe the job of the system administrator in IT to be one of the most important, if not the most important. System administrators must design, implement, and maintain computer systems. This is obviously one gigantic chunk of what makes up the information technology field as a whole.
It has often been my experience that the sysadmin(s) for an organization is/are the best informed resources from an IT perspective (at least if you're a good one). Who else do you talk to when needing to discuss any significant change to an organization's computing infrastructure?
To the person who commented that there was no mention of good troubleshooting skills as qualification for a good sysadmin....I believe that fell under the comment that a component of the sysadmin's job was to keep the systems running. To be able to troubleshoot and solve problems is a prerequisite to keeping systems running.
Second, "strong experienced based opinions" is crap.
It's better than just 'strong opinions'. Anyone logical enough to realize that you should normally have opinions based on experiences is normally logical enough to be reasoned with regarding how those experiences may differ from other experiences, and how 'new' approaches may in fact be better.
In your Air Force situation, it sounds like the people you were dealing with had had little or no experience with the type of topology you were recommending.
creation science book
Life in the trenches: a sysadmin speaks
:-). My partner's response is: "It's good, it keeps him out of my hair while I'm programming". Actually, we both work in the Internet industry. Her skill set is slightly different to mine. She's better at programming and much better at management tasks, whereas I'm better at systems administration and don't have much interest at all in taking on management roles.
By Sam Varghese
December 27 2002
Craig Sanders: "A sysadmin who doesn't have strong experience-based opinions about how things should be done probably isn't very confident in their own ability to do the job."
As recently a decade ago, a systems administrator wasn't really needed in every medium- or large-sized corporation. There were motley assemblages of computers which were used for this task and that and if one or two broke down, then the supplier came in and fixed them.
But as use of the Internet spread, offices began to be increasingly networked, servers appeared in numbers and men and women were needed on-site to keep these metallic objects - which had slowly assumed tremendous importance as data repositories - going. Uptime became important.
Early on, the men and women - and lots of pimply-faced teenagers - who took on these jobs were considered a breed apart. They weren't exactly flavour of the month - and seemed to return the compliment by sticking to themselves as much as possible.
But as geeks became more and more socially accepted, it came to be known as a cool profession - though most people never knew what these IT folk really did.
Some migrated to this line out of a genuine liking for what they would be doing; as the tech boom gathered momentum, many others with dollar signs in their eyes joined what looked like a never-ending job queue.
advertisement
advertisement
Craig Sanders belongs to the former category. Around the time when IBM put out its first PC, he was already working as a programmer - at 14.
In 1982, he went into a support/sysadmin role and has stayed in that line ever since. Says he: "I guess this job was inevitable for me since I discovered computers at the age of 11. The only job I've ever had that wasn't in the computer industry was a brief stint selling hotdogs outside a pub while I was at university, which lasted until I found a part-time programming job."
From the early 1990s onwards, Sanders began to focus on Unix systems administration almost exclusively. From 1994, his focus has been Linux. He is a developer for the free Linux distribution, Debian.
Sanders currently works at Vicnet, an Internet Service Provider focusing on community groups and libraries. He started as a systems administrator in November 1997, and was promoted to senior systems administrator a year or so later. Most of the Vicnet servers run Debian GNU/Linux; some run Sun Microsystems' Solaris operating system, and there are also a few Windows NT servers.
Sanders inspires strong emotions - he is convinced about what he believes in and does not suffer fools gladly. He is forthright in his opinions but is rarely technically challenged on them. He is probably one of the few 35-year-olds in the country who until recently did not have a television set because he hates advertising. He now has one but uses it only to watch the news on the ABC and DVDs.
He was interviewed by email.
What are your fundamental tasks as a sysadmin?
To keep the systems running.
To plan and implement upgrades and new services.
To plan for disaster, minimising the risk and the potential damage, including backups and disaster recovery planning.
To resolve any systems problems that crop up or, better yet, to see the warning signs and head them off before they become a problem.
To keep my skills up-to-date.
To be a knowledge resource for the company.
What qualities do you rate as essential for a good sysadmin?
In rough order of importance:
Aptitude.
Ability to learn and understand complex subjects quickly.
Ability to hold a mental model of How Things Work.
Caution and knowing how to make changes in a way that you can quickly and easily undo if you need to i.e. revision management skills.
Communications skills - you need to not only know something, you need to be able to explain it to others in plain English so that reasonably intelligent non-experts can understand it.
Note that training and formal qualifications aren't on that list. They're useful, but only in addition to the above traits, not as a substitute for them.
Sysadmins are often accused of being control freaks. They are also accused of being vengeful people, who use their technical knowledge to harass users and keep upper management in check. Your comment?
I can understand why some people might feel this way, but I don't agree. There is an inherent tension between maintaining a system's current functionality and developing new functionality. Part of a sysadmin's role is to manage the impact of development projects so that they don't negatively affect the existing systems. This is often interpreted as being adversarial.
A sysadmin has to know not only what can be done but also what cannot (or should not) be done. Sometimes that means stopping people from doing the wrong thing and sometimes it means making sure that they do the right thing. This can annoy people or lead them to believe that they are being deliberately thwarted, but it's really just the sysadmin doing the job they were hired to do.
It's difficult to put it in more general terms than that, because it is highly situational - for most tasks, there are several ways to do it. Some ways are obviously better and anyone can see them; others are not so obvious, it requires a lot of experience to be able to foresee how subtle differences and even subtler interactions between different components can have an enormous impact on the final outcome; and some ways are obviously wrong to an experienced tech but may appear to be right to someone blinded by glossy marketing brochures or a slick sales-pitch for whatever the latest snake-oil buzzword is.
Also, a sysadmin who doesn't have strong experience-based opinions about how things should be done probably isn't very confident in their own ability to do the job... and if they're not confident, why should you be? Sometimes this strength of will and confidence may be interpreted as being a "control freak", especially by people who don't have the background to understand the reasons why a sysadmin has made particular decisions.
Does life as a sysadmin really end after you leave work? Or are you on edge, waiting for your mobile to ring?
The job never really ends, but I'm certainly not on edge. I'm on call 24/7 but if I've done my job right I generally don't have to worry about being called in the middle of the night.
Have you ever been in the position where you had to act as mentor for someone in this line? If so, how did you go about it?
Yes, I have had (and still have) several junior system admins. Part of my job is to train them. I do that by setting an example, setting standards (e.g. of quality) for how things should be done, teaching them how to do something and, most importantly, teaching them how or why it works. Then I gradually give them responsibilty for their own systems or service areas.
I think that having a good understanding of how something works is far more valuable than having a specific rote procedure to follow. If you understand it, you can deal with situations that haven't been pre-scripted i.e. you can deal with unplanned emergencies. If all you know is a set of rote procedures then you're in serious trouble when something crops up for which you don't have a set procedure.
What's been the biggest crisis you've faced as a sysadmin? How did you resolve it?
The worst disaster I can recall was when a rack shelf fell apart (the builder put it together the wrong way) and dropped a few servers on the floor from about two metres high. One of our Web servers died, the disk heads crashed. I had to build a replacement from spare parts and restore the data from backup. It was back up and running the same day, and we only lost a few hours worth of Web server log files.
Do you find that your IT involvement cuts you off from people? Has it affected you in any way?
No, not really. I have noticed that until the Internet became popular in the mid-90s it was social death to admit to any interest in computers, and it was certainly not acceptable to talk about them at parties. That's changed now. It's still considered "geeky" but it's not the unforgivable social crime that it once was. You still have to pretend not to know much about computers, but these days it's so you don't waste the entire party solving someone's computer problems for them.
I think, though, that to be any good at this job you have to have a particular way of thinking and looking at the world. For those who like personality tests, Myers-Briggs personality types INTJ and INTP typically make good systems admins. These personality types are fairly uncommon (less than five percent of the population), and the worldview is moderately alien to most people... so, while there may be some level of "cut off" from other people, the job isn't the cause.
This is not to say you have to be INTJ/INTP to be a good system admin, just that the percentage within sysadmin and related professions is many times higher than the percentage within the general population.
What is your partner's reaction to the line you have chosen (and love)?
The flippant answer is that I solved that problem by training her to be a systems administrator too
How much input would a good sysadmin have into choice of platforms in a company? Or is this solely a matter for management?
Management should set the budget and the overall needs. Systems staff need free reign to implement a solution that meets those needs within the budget.
Otherwise, what you end up with is a system that doesn't work very well because it was designed by people who are not qualified to design it. Managers are skilled at management tasks, they know what the business needs of the company are but, as a general rule, they do not have the knowledge or experience required to make technical decisions.
In my experience, it's an iterative process where management sets the budget and outlines the requirements. The sysadmin does the research and comes back with a list of options that may meet those needs, detailing the pros and cons of each option. A few rounds of this narrows down the options under consideration until only one or two are left. Then a decision is made and implementation planning begins.
How would you go about introducing new technology in a company - stuff which you know will make life easier for both users and admins but which has no support from a management team which views change as disruptive?
As a general rule, it's best to talk about feature sets and not about particular brands of technology. That's a good way to look at it anyway, because a good design is modular and any component should be easily replacable by a similar component that does the same job.
I guess you're asking about Linux and other Open Source software here, so I'll use Linux and Samba as an example: when a need comes up for a new file or print server, don't talk about installing a Linux box, talk about installing a new file or print server. As long as what you implement does the job and works reliably, no one will care how it's done as long as it works.
Otherwise, a generally cautious approach is the best way. Don't introduce sweeping changes, overnight - migrate to them gradually. start with small narrowly-defined services, e.g. take some of the workload off your NT file server by adding a Linux print-server or two (you can do this at effectively no cost by recycling an obsolete desktop machine). or protect your MS Exchange server by hiding it behind a firewall and using Linux and postfix as a safe, anti-spam, virus-scanning email gateway between Exchange and the Internet.
And finally, you need to be able to recognise when it isn't a good idea to change something. even though the new technology may be better, the workflow and routine of your site may be too closely tied to the existing product. No amount of superior technology is going to justify disrupting a routine that works. If you can introduce the new technology without disruption, then do it. Otherwise, don't.
What's your biggest complaint about the profession?
I don't have much to complain about. I like the job, I enjoy the challenges, and I get a real sense of accomplishment from making sure that the systems I'm responsible for work reliably 24 hours a day, 7 days a week.
The biggest issue would be that often there is no clear distinction between work and non-work hours - it's very easy to work 12 or 15 hours or more per day when you have a difficult or interesting problem to work on.
This is true for the job in general, but telecommuting makes it even more so. OTOH, (on the other hand) telecommuting is one of the major benefits of the job.
And the biggest plus point?
Telecommuting. I can do at least 70 percent of my job from home at any hour of the day or night. With appropriate encryption, it makes no difference whether I am sitting at the console or at my desk at the office or at my desk at home - or anywhere for that matter.
I've logged in to my systems at work while away at conferences and fixed things. I've even logged in from an Internet cafe while on holiday in India, although the lag on that link was too slow to get much done.
Final words?
Systems Administration is the kind of job that nobody notices if you're doing it well. People only take notice of their systems when they're not working, And they tend to forget that a lot of work and expertise goes into making sure that they continue working.
But that's as it should be - computer networks are infrastructure that you should be able to rely on, to take for granted, just like telephones and electricity. If you can't do that, then there's something wrong, something that can and should be fixed.
Systems Administration is the kind of job that nobody notices if you're doing it well. People only take notice of their systems when they're not working, And they tend to forget that a lot of work and expertise goes into making sure that they continue working.
You only ever talk about IT when things go wrong. In my mind, that's a thankless job. I am SO thankful that there are people that don't mind that... And this guy is a professional through and through:
But that's as it should be - computer networks are infrastructure that you should be able to rely on, to take for granted, just like telephones and electricity. If you can't do that, then there's something wrong, something that can and should be fixed.
I like how he takes responsibility. This is unbelievable. I want him as my IT guy now.
a sysadmin has to be _ethical_. They're in a position to witness alot of people's private information, especially in a place like an ISP - not even Echelon can monitor people online like the sysadmin can.
I also fully agree that when you're on vacation, if your underlings can't keep the ship together, you're not doing a very good job.
What he doesn't hit on very well in his preachy missive is the importance of diplomacy. I work in a big enough operation that I don't even deal with the end-customers, we have an application support team for that. This means that (a) the problems are reduced, since I only have to worry about a handful of real "users" who can damage the systems and (b) the problems are greater, because those guys are vastly better at really kicking the legs out from under my boxes! So it's mightily important to always touch base with the application support teams, and keep a continuous stream of communication up. It's easy to lose that, especially in a giant operation, especially when your specialty is copping an attitude.
And finally: Why do so many sysadmins dedicate their lives to looking like freaks? Find a shower, a razor, a comb, and use them, people!
He's dead-on with his observation that personality type and aptitude are the most important qualities in a sysadmin. I am fighting a battle with a boss who actually thinks you can train someone (anyone) to be a sysadmin. Unfortunately when these people fail miserably I get accused of poor training. Oh well, I can always work for a service provider in my next life.
What's your biggest complaint about the profession?
I don't have much to complain about
HUH!? I'm gonna go out on a limb here using my expereince and the people I know and say this is the exception and not the norm... Is this guy for real? Every sysadmin professional I know complains about the users, the hours, the pay and their job security. And what's this Telecommuniting BS? 70% of the time he was able to stay at home? Am I missing something here? This does NOT sound like the average Sysadmin Job I've come to know. Most employers are too damn anal for that to occure, even if you could effectively...
Jeez... I must be missing something here... Talk about a raw deal...
You need a FREE iPod Nano
> hell" mentality, he's just a little more polite about it.
Too true and still an unfortunate stereotype of all too many self-annointed sysadmins, or at least those who can get away with this attitude. Unfortunately, many inexperienced management types still think that this is acceptable behavior - but that is changing.
He sounds like he works at a relatively small and fairly autonomous site without too much interaction with other groups/departments using the systems on a day-to-day basis. His management also doesn't appear to know what is going on - but it probably doesn't matter and they don't care given the circumstances of this particular site.
Any one involved in system admininstration or interested in this type of job should consider the recent book "The Practice of System and Network Administration" (by Thomas A. Limoncelli and Christine Hogan) a must read. This is a far more realistic description of contemporary practices in system administration than the comments made in this article.
As recently a decade ago, a systems administrator wasn't really needed in every medium- or large-sized corporation. There were motley assemblages of computers which were used for this task and that and if one or two broke down, then the supplier came in and fixed them.
But as use of the Internet spread, offices began to be increasingly networked, servers appeared in numbers and men and women were needed on-site to keep these metallic objects - which had slowly assumed tremendous importance as data repositories - going. Uptime became important.
Early on, the men and women - and lots of pimply-faced teenagers - who took on these jobs were considered a breed apart. They weren't exactly flavour of the month - and seemed to return the compliment by sticking to themselves as much as possible.
But as geeks became more and more socially accepted, it came to be known as a cool profession - though most people never knew what these IT folk really did.
Some migrated to this line out of a genuine liking for what they would be doing; as the tech boom gathered momentum, many others with dollar signs in their eyes joined what looked like a never-ending job queue.
Craig Sanders belongs to the former category. Around the time when IBM put out its first PC, he was already working as a programmer - at 14.
In 1982, he went into a support/sysadmin role and has stayed in that line ever since. Says he: "I guess this job was inevitable for me since I discovered computers at the age of 11. The only job I've ever had that wasn't in the computer industry was a brief stint selling hotdogs outside a pub while I was at university, which lasted until I found a part-time programming job."
From the early 1990s onwards, Sanders began to focus on Unix systems administration almost exclusively. From 1994, his focus has been Linux. He is a developer for the free Linux distribution, Debian.
Sanders currently works at Vicnet, an Internet Service Provider focusing on community groups and libraries. He started as a systems administrator in November 1997, and was promoted to senior systems administrator a year or so later. Most of the Vicnet servers run Debian GNU/Linux; some run Sun Microsystems' Solaris operating system, and there are also a few Windows NT servers.
Sanders inspires strong emotions - he is convinced about what he believes in and does not suffer fools gladly. He is forthright in his opinions but is rarely technically challenged on them. He is probably one of the few 35-year-olds in the country who until recently did not have a television set because he hates advertising. He now has one but uses it only to watch the news on the ABC and DVDs.
He was interviewed by email.
What are your fundamental tasks as a sysadmin?
What qualities do you rate as essential for a good sysadmin?
In rough order of importance:
Note that training and formal qualifications aren't on that list. They're useful, but only in addition to the above traits, not as a substitute for them.
Sysadmins are often accused of being control freaks. They are also accused of being vengeful people, who use their technical knowledge to harass users and keep upper management in check. Your comment?
I can understand why some people might feel this way, but I don't agree. There is an inherent tension between maintaining a system's current functionality and developing new functionality. Part of a sysadmin's role is to manage the impact of development projects so that they don't negatively affect the existing systems. This is often interpreted as being adversarial.
A sysadmin has to know not only what can be done but also what cannot (or should not) be done. Sometimes that means stopping people from doing the wrong thing and sometimes it means making sure that they do the right thing. This can annoy people or lead them to believe that they are being deliberately thwarted, but it's really just the sysadmin doing the job they were hired to do.
It's difficult to put it in more general terms than that, because it is highly situational - for most tasks, there are several ways to do it. Some ways are obviously better and anyone can see them; others are not so obvious, it requires a lot of experience to be able to foresee how subtle differences and even subtler interactions between different components can have an enormous impact on the final outcome; and some ways are obviously wrong to an experienced tech but may appear to be right to someone blinded by glossy marketing brochures or a slick sales-pitch for whatever the latest snake-oil buzzword is.
Also, a sysadmin who doesn't have strong experience-based opinions about how things should be done probably isn't very confident in their own ability to do the job... and if they're not confident, why should you be? Sometimes this strength of will and confidence may be interpreted as being a "control freak", especially by people who don't have the background to understand the reasons why a sysadmin has made particular decisions.
Does life as a sysadmin really end after you leave work? Or are you on edge, waiting for your mobile to ring?
The job never really ends, but I'm certainly not on edge. I'm on call 24/7 but if I've done my job right I generally don't have to worry about being called in the middle of the night.
Have you ever been in the position where you had to act as mentor for someone in this line? If so, how did you go about it?
Yes, I have had (and still have) several junior system admins. Part of my job is to train them. I do that by setting an example, setting standards (e.g. of quality) for how things should be done, teaching them how to do something and, most importantly, teaching them how or why it works. Then I gradually give them responsibilty for their own systems or service areas.
I think that having a good understanding of how something works is far more valuable than having a specific rote procedure to follow. If you understand it, you can deal with situations that haven't been pre-scripted i.e. you can deal with unplanned emergencies. If all you know is a set of rote procedures then you're in serious trouble when something crops up for which you don't have a set procedure.
What's been the biggest crisis you've faced as a sysadmin? How did you resolve it?
The worst disaster I can recall was when a rack shelf fell apart (the builder put it together the wrong way) and dropped a few servers on the floor from about two metres high. One of our Web servers died, the disk heads crashed. I had to build a replacement from spare parts and restore the data from backup. It was back up and running the same day, and we only lost a few hours worth of Web server log files.
Do you find that your IT involvement cuts you off from people? Has it affected you in any way?
No, not really. I have noticed that until the Internet became popular in the mid-90s it was social death to admit to any interest in computers, and it was certainly not acceptable to talk about them at parties. That's changed now. It's still considered "geeky" but it's not the unforgivable social crime that it once was. You still have to pretend not to know much about computers, but these days it's so you don't waste the entire party solving someone's computer problems for them.
I think, though, that to be any good at this job you have to have a particular way of thinking and looking at the world. For those who like personality tests, Myers-Briggs personality types INTJ and INTP typically make good systems admins. These personality types are fairly uncommon (less than five percent of the population), and the worldview is moderately alien to most people... so, while there may be some level of "cut off" from other people, the job isn't the cause.
This is not to say you have to be INTJ/INTP to be a good system admin, just that the percentage within sysadmin and related professions is many times higher than the percentage within the general population.
What is your partner's reaction to the line you have chosen (and love)?
The flippant answer is that I solved that problem by training her to be a systems administrator too :-). My partner's response is: "It's good, it keeps him out of my hair while I'm programming". Actually, we both work in the Internet industry. Her skill set is slightly different to mine. She's better at programming and much better at management tasks, whereas I'm better at systems administration and don't have much interest at all in taking on
management roles.
How much input would a good sysadmin have into choice of platforms in a company? Or is this solely a matter for management?
Management should set the budget and the overall needs. Systems staff need free reign to implement a solution that meets those needs within the budget.
Otherwise, what you end up with is a system that doesn't work very well because it was designed by people who are not qualified to design it. Managers are skilled at management tasks, they know what the business needs of the company are but, as a general rule, they do not have the knowledge or experience required to make technical decisions.
In my experience, it's an iterative process where management sets the budget and outlines the requirements. The sysadmin does the research and comes back with a list of options that may meet those needs, detailing the pros and cons of each option. A few rounds of this narrows down the options under consideration until only one or two are left. Then a decision is made and implementation planning begins.
How would you go about introducing new technology in a company - stuff which you know will make life easier for both users and admins but which has no support from a management team which views change as disruptive?
As a general rule, it's best to talk about feature sets and not about particular brands of technology. That's a good way to look at it anyway, because a good design is modular and any component should be easily replacable by a similar component that does the same job.
I guess you're asking about Linux and other Open Source software here, so I'll use Linux and Samba as an example: when a need comes up for a new file or print server, don't talk about installing a Linux box, talk about installing a new file or print server. As long as what you implement does the job and works reliably, no one will care how it's done as long as it works.
Otherwise, a generally cautious approach is the best way. Don't introduce sweeping changes, overnight - migrate to them gradually. start with small narrowly-defined services, e.g. take some of the workload off your NT file server by adding a Linux print-server or two (you can do this at effectively no cost by recycling an obsolete desktop machine). or protect your MS Exchange server by hiding it behind a firewall and using Linux and postfix as a safe, anti-spam, virus-scanning email gateway between Exchange and the Internet.
And finally, you need to be able to recognise when it isn't a good idea to change something. even though the new technology may be better, the workflow and routine of your site may be too closely tied to the existing product. No amount of superior technology is going to justify disrupting a routine that works. If you can introduce the new technology without disruption, then do it. Otherwise, don't.
What's your biggest complaint about the profession?
I don't have much to complain about. I like the job, I enjoy the challenges, and I get a real sense of accomplishment from making sure that the systems I'm responsible for work reliably 24 hours a day, 7 days a week.
The biggest issue would be that often there is no clear distinction between work and non-work hours - it's very easy to work 12 or 15 hours or more per day when you have a difficult or interesting problem to work on.
This is true for the job in general, but telecommuting makes it even more so. OTOH, (on the other hand) telecommuting is one of the major benefits of the job.
And the biggest plus point?
Telecommuting. I can do at least 70 percent of my job from home at any hour of the day or night. With appropriate encryption, it makes no difference whether I am sitting at the console or at my desk at the office or at my desk at home - or anywhere for that matter.
I've logged in to my systems at work while away at conferences and fixed things. I've even logged in from an Internet cafe while on holiday in India, although the lag on that link was too slow to get much done.
Final words?
Systems Administration is the kind of job that nobody notices if you're doing it well. People only take notice of their systems when they're not working, And they tend to forget that a lot of work and expertise goes into making sure that they continue working.
But that's as it should be - computer networks are infrastructure that you should be able to rely on, to take for granted, just like telephones and electricity. If you can't do that, then there's something wrong, something that can and should be fixed.
troubleshooting often has nothing whatsover to do with the system at all.
The primary difference between a really good admin and a BOFH is the realization that "lusers" are *part of the system.* A really, *really* good admin has to be that apparently rarest of geeks, the person with outrageously good technical *and* people skills.
After all, the admin isn't just responsible for the machines, he is also the primary interface between the machines and the people.
How do you know if your company has a really talented admin? If he kills all of a user's processes and deletes all of his files, and the user is so greatful the treats the admin to lunch.
Now *that* is evidence of an admin who has figured out what his job is and how to do it. Which is, unfortunately, rare.
KFG
He even LOOKS the part...
"Consider how lucky you are that life has been good to you so far. Alternatively, if life hasn't been good to you so far
Jesus, this guy could be RMS' brother. Or sister, for that matter. Why do these types have no regard for personal hygene or grooming. It's really sad, these are supposedly intelligent people yet they all seem to be incapable of tidying themselves.
Note to future sysadmins:
Get a hair cut.
Dress neatly in clean clothes that are not from the previous decade.
Bathe! Frequently!
And brush your God damned teeth, at least twice a day!!!
impeccabully dressed payper liesense peddling stock markup ?pr? eyecons? IT would be better to be almost anything besides won of those.
I agree entirely.
I've found that working with people that have "strong experienced based opinions" is a painful experience. These are generally geeky people that complain every time management asks them to do anything other than install the latest version of the OS.
That type of person is a pain to manage and even worse to work with. You need to realize that management is there for a reason, to direct the company. You are there to FOLLOW INSTRUCTIONS, not spout back on how things should be done because you are an "experienced expert".
Being competent keeps you from getting fired, but that attitude keeps you from getting anywhere in the company (and makes management WISH they could fire you).
Yes, you are missing out. A good sysadmin at a decent company can have a very good life. I have had sysadmin positions with small, medium and very large companies where I telecommuted 90% of the time. In one job I telecommuted 100% of the time for a year, before I felt a bit lonely and started frequenting the coorporate campus for a few hours a week. It's amazing what a difference there is when people can put a face with the voice at the other end of the phone.
I was a good sysadmin and I have greater aspirations than this guy does so, I have moved up and beyond these older jobs but, they were very good jobs while I was there.
You're missing out. The question you must ask is, why? Are you really as good at your job as you think you are? Are you able to relate to management or are you constantly trying to win pissing contests with them? Do the users like you, or do the fear or view you with disdane? Honest answers to these questions are harder to get than you might think. You may want to ask a peer or higher-up engineer type for brutally honest answers to these questions. Engineer types will usually oblige, provided they aren't close friends or subordinates. Once you have these answers, accepting them and working to truely address potential shortcomings could completely turn things around for you. Good luck.
Because most people can do some of these things, they can end up doing sysadmin work. Does that make someone a sysadmin? I have interviewed for sysadmin roles before and always been amazed at the people who have used an application, or watched and install, and then applied for the sysadmin job. It's not enough.
The problem is, lots of people doing this kind of work without the training and experience (and often, no mentor either - nontechnical boss) give the profession a bad name - hence the whole BOFH subculture.
This link describes some of the issues related to this job that isn't very mature at all ...
Why did you buy the computer? To run programs. And so step forward the programmer...
Why did the programmer write the program? Because it performed the task needed. And so step forward the analyst...
Who needed the task performed? And so step forward the end-user...
I've always thought Syadmins to have an over-inflated importance in the world. As I show above, I put them third or fourth in the pecking order (depending on whether the end-user and the analyst are not the same people). Many admins forget that the point isn't to have lots of wonderfully run locked-down computers that don't do anything (damned users! get in the way of my policies...). A computer is a tool - a beautifully polished tool that doesn't do anything is worthless.
Cheers,
Ian
Australia? I thought they were still doing mainframe isp and walkabouts. Who knew that they knew what computers were?
I will agree with someone else who posted that this guys comments about personality types are right on. You do not *have* to have a particular personality type to be a good sysadmin, but you need to at least have the self-awareness to know what your personality is and how it affects your job performance.
Of course people on slashdot are always looking for something to disagree with, so a few of you have already lashed out at the "strong experience-based opinions" quote. Experience is the number one most important part of being good at *any* job. If you don't agree, then you probably don't have enough experience.
I'll also say this: You don't have to agree with everything someone says to learn from them. (In fact, if you only listen to people who you are in complete agreement with, you will never learn much of anything.) There are a lot of good points in this article, and even if you are somehow offended by the experience-based opinions remark or something else, you can still gain something from it.
Here's a positive comment.
I thought that was an insightful article. System administration is the process of keeping together an organization's information infrastructure. People often find this job to be non-human oriented, but it is in fact completely human oriented. The good sys admin is constantly thinking of, and even torturing themselves over how the users will be affected by anything he/she ever does and how it can make their lives easier.
The really good sys admins will unfortunately be perceived as adversaries because they would rather disagree and cause a political stir than develop a system that they believe is going to harm the users more in a long run.
Most intelligent people can figure this out, and will respect their sys admin's position in the company. The sys admins who stay quiet during meetings when they see the company making a wrong move are the ones who don't care, and IMO better fit the profile of BOfH.
At the heart of the matter, our profession is to increase the quality of life through information technology. Anyone who doesn't see their IT profession this way is in the wrong career.
As long as all the SysAdmins seem to be making it up as they go along, we will continue to be marginalized and geek-ified by management. Try on for size:
- ITIL (IT Infrastructure Library)
- More ITIL
- IT Service Management Forum (the U.K. headquarters)
- CMM (Capability Maturity Model) Technical Engineering Practices
- The Open Group's Technical Architecture Framework.
Heck, even Microsoft is trying to get into the picture with its Microsoft Operational Framework, a kind of embrace-and-extend on ITIL, though I don't know of many places that are actually using it.It's not that the SysAdmin necessarily has to manage these processes - though in a small shop no one else will - but he/she/it needs at least to be able to talk the language and understand the processes that the IT Manager has set up. And if you are managing the shop, then this is your job. You must know this stuff as a matter of professional responsibility and "keeping up" in your field.
A 20 min. presentation to the other managers on Best Practices and Processes in IT Management will gain you a lot of credibility and help lift you out of the geek gutter. There are decades worth of lessons that have been learned the hard way and documented into these processes. When you can demonstrate to management that you are drawing on a substantial body of knowledge that is geared towards improving service and reducing total cost of ownership, you will gain their respect (assuming that you care about their respect).
Beyond this, I want to emphasize an excellent point that Sanders makes in the article. The SysAdmin job is one that is invisible if you're doing it right. A good day at work is a boring day. Excitement is a sign that something has gone wrong. You should structure your environment to be as boring and reliable as possible.
Too many SysAdmins live off the adrenaline rush of fixing a broken server while everyone else in the organization sits on their thumbs waiting. That's costly for the organization, but ironically is the easy way out for the SysAdmin - you don't need to be disciplined or structure your time or do any planning or thinking, just jump from crisis to crisis. It's much more challenging to turn it into a boring desk job where most of your work is pushing paper and the machines pretty much take care of themselves. But guess which option is better for the organization's mission?
Once you do get to that Nirvana state of boring life, you can strategize how to produce some measurables so you can blow your department's horn at the monthly managers meeting. Because if you do your job well, with the result that your work is invisible, they'll cut your funding unless you keep in their face on a regular basis.
.nosig
One of the best desciptions of what I would say a SysAdmin has to be.
While I did not read the word "proffesionalism" anywhere in the article, and I feel that it should be there, his attitude to the job definitly shows that he is a proffesional...
I will encourage a few people to read this article.
I think that having a good understanding of how something works is far more valuable than having a specific rote procedure to follow. If you understand it, you can deal with situations that haven't been pre-scripted i.e. you can deal with unplanned emergencies. If all you know is a set of rote procedures then you're in serious trouble when something crops up for which you don't have a set procedure.
As another poster mentioned here, his number one quality for the job is aptitude. If that's not problem solving, I'm not sure what is. So it seems that you and the article agree, except that the author expects his juniors to get it and would not keep them around long if they did not.
Friends don't help friends install M$ junk.
Ye gods, how true!
--K.
Sig: Bad people happen. Try to avoid being one of them.
The reason why it's not a thankless job for Craig Sanders is because he is in a worthwhile position within his company, able to control and hence take pride in the running of complete systems, not employed as a mere grease monkey without input yet always blamed when the systems are down.
I think many sysadmins on this forum will find that the following rings a bell. You begin with total control in a startup IT team, decide on and bring into operation all aspects of a solution and keep it all running perfectly for years, with near-zero downtime and great job satisfaction. Then the corporate machine takes over, basically overturns everything you've done and creates an absolute disaster, and despite ignoring utterly all your input, you are to blame since you're the sysadmin. Needless to say, job satisfaction is, let's just say, less. This ring a bell?
Craig Sanders has managed to avoid stage 2 so far. He deserves only praise, in my book.
"The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
Why did you buy the computer? To run programs. And so step forward the programmer...
Of course programmers/analysts are another component of IT. Unfortunately, many programmers typically are only aware of the programming language they are using and the business requirements they are attempting to fulfill. Most are spotty in terms of their overall computing knowledge. We are, after all, talking about , and I stress this, information technology, and not business requirements. Many programmers I've encoutered have a weak overall knowledge of information technology. I guess I say this as I was a developer and DBA for the first 5 years of my involvement in IT, and I definitely see a vast difference between the typical programmer and the typical sysadmin. You often see a sysadmin who also knows how to program in several languages, yet rarely see a programmer how also knows how to design, implement, and maintain systems based around multiple hardware platforms and operating systems.
Who needed the task performed? And so step forward the end-user...
The end-user plays little to no role in information technology. They use the program, programmed by the programmer, running on computing resources designed and maintained by the sysadmin.
Now, I think the difference in our viewpoints is caused by what we are evaluating. I'm evaluating the importance of a sysadmin in the information technology realm. It appears you are evaluating these differing roles in terms of a company as a whole (not just IT). And from the viewpoint you are taking, I agree with you.
I have been administering systems for over a decade now. I do many of the technical interviews for the company I work for... or at least I did when we were hiring :( . Dismiss me if you want, I don't particularly care, but be aware it may be me or somebody much like me, on the other end of the phone the next time you try and get a job.
For everyone whining about the fact that he says a good sysadmin should have strong opinions based on experience... If you think that every problem is going to be so clear cut and so clean that you can just bang out an optimal solution and provide a clean and mathmatical defense for it, all you have done are home or academic excercises.
The problem domain for solutions is so incredibly broad, and so incredibly rich, that if you are not depending on collection of good solid abstract rules of thumb and effective practices, you will never get to a good solution. You have to use intuition to narrow down the problem domain to a few concrete approaches, and then apply logic and experience to decide which of them to implement and how.
These are not opinions like "NT Sucks, Linux rules", these are opinions like "I don't want to hinge my business case on an operating system controlled by a single vendor". I don't want an enterprise IT infrastructure that depends on technology that only runs on non-scalable hardware". "I don't want an operating system that I cannot remotely administer". "I want an operating system that allows me to update and maintian, stop, and start some subsystems without effecting other subsystems". "I want an operating system where I can apply security patches without being forced to install operating system updates". You get the idea.
Having an open mind is important, but at some point you have to get off your ass and decide something, and act upon that decision. The older I get, the more important I have realized this becomes.
A group of people with "strong opinions based on experience" can get together and hammer out a list of pro's and cons, and come up with an excellent solution to a problem, fully aware of what the solution does well and where it will be weak. It will be a stressfull meeting, and tempers may occasionally flare, but when you finally grind through it you will end up on solid ground, and everyone will likely be on board.
A bunch of people with "open minds and no strong opinions" are going to dither about endlessly and end up with an unfocused, innefective, designed by committe monstrosity.
Acedemia is all about exploration and investigation. Work is about getting things done. Note though that even the academia people typically won't get much "exploration" done if their home made router is down because it is an old Linux box built around a $20 commodity power supply that just went up in smoke, and the only guy that knew how to set up the IPTables to get the routing right left to go to grad school 3 months ago.
I am with this guy... a lack of a strong opinion and the ability to defend it, suggests to me a lack of experience. How on earth can you do something day in and day out, sweat over it, bleed over it, live and die by it, day by day and year by year, and not form an opinion?
Mathematically impossible requirements are technically not against policy.
After reading the replies, I can say one thing with certainty. Every sysadmin knows more than every other sysadmin. BTW take the personality test, it's fast, and pretty cool. (ISTP)
hired out are a special case. Certainly an ISP is the most obvious example, and one where the indirection is so great most users don't even realize they're users.
I'd only point out that help desk people are themselves users of the system, and generally rank only a smidgeon above subscribers on the "luse-O-meter."
My point stands.
KFG
I've been a sysadmin for the past 5 years, two of them at a large department in a very big educational institution. I have to say that of all jobs I've had in the past, this is the most personally unrewarding.
Sure, the pay is good, and the benefits are nice, and you get to sit in your comfy chair most of the time punching buttons and not really doing anything in particular. However, this "bliss" comes with the following drawbacks:
I was an education major in college, and during one of the classes our professor told us: "when you start teaching, there will be rich schools and poor schools. If you work in one of the rich schools you will have a good salary, good budget, nice classrooms, and decent lunches. If you work in a poor school, you will have none of that, plus drugs, violence, and complete lack of parental involvement. Believe it or not, some people prefer to work in poor schools simply because if they are doing their job well, there will be people who will stop them every day in the hallway and tell them how much they admire their work. Not only that, but people working in poor schools are able to see with their own eyes how much difference they are making in the lives of the children they teach."
That seemed weird to me then, but now I think I understand. It all comes down to what one thinks to be a good reward for their work. If it is good pay, quick career path, and a Porshe by the time you're 30, then being a sysadmin is your dream job (granted, of course, that you're good at it). However, if you are looking for something that is personally rewarding, something you want to feel good about doing... You might want to pick a different carreer. Or at least do it only until you start feeling burnt-out.
Me? Oh, I'm quitting as soon as I can afford it. :)
If you open yourself to the foo, You and foo become one.
That's the funniest thing I've ever read on Slashdot. And it's funny because it's true. ;-)
Problem solving ability would probably be included in "aptitude."
> I wouldn't say that. He probably missed the machines...
In that case, he misses them a bit TOO MUCH ;-)
(There must be some sort of treatment for people with this type of psychological profile - those who just can't let go ... Think about it, he'd rather call his computers while on vacation rather than his human IT staff!)
The article includes a link to Understanding your Myers Briggs Personality type just before it states that "INTJ and INTP typically make good systems admins"
I'm an INTP, and I never realised it. So I read up on the other types (table of links at the bottom of the page). Fasinating. I shudder at the thought of waking up some day as an EFSJ.
I don't know if the slashdot poll code can handle 16 options, but I suspect that the "fairly uncommon (less than five percent of the population)" people are probably more like 90 around here. Perhaps it is time to find out.
The most truthful point is that nobody ever remembers when you're right, only when you're wrong. This is probably why I've not had a raise for some time.
I came out ISTP as well, I'm going to take that test to my philosophy class when I go back to HS tommorow. *sigh*
Then, after that, I get to go play...errr...work on the school computers, and get credit for it! So much play work to do.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Don't do what I've done...all my friends & family know I fix computers too, so I get all the questions from them! Stay silent, or be forever their helpdesk!
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
(places hand to ear)
I think I can hear a Dingo eating your DNS server.
wakka wakka
Curious, how much did this site sell for? How many users are logged in normally at once? Thanks!
Actually, he works for VicNet, which is basically an ISP whose sole role seems to be to make it irritatingly difficult to achieve anything at all while their systems are operating. Fortunately their systems don't operate terribly well, so this is a minor problem in practice. They've always been a classic example of the type of ISP run by hairy unhygienic fascists with the ISO Standard BOFH mentality and no real interest in going outside their comfort zones. Best avoided. So this interview seems spot on: just don't kid yourself that all sysadmins are like this git.
I did see one about 5 years ago;
Wanted: Web master. 10 years experience.
I drank what? -- Socrates
Look at this guy... he looks like a mess!
I guess this job was inevitable for me since I discovered computers at the age of 11.
He says that as if that's a particularly early age to screw around with computers!
</sarcasm>
mogorific carpentry experiments
#1 If users are made to understand that the system administrator's job is to make the computers run, and not to make users happy, they can, in fact, be made happy most of the time.
#2 If users are allowed to believe that the system administrator's job is to make users happy, they can, in fact, never be made happy. Futhermore, in their quest for happiness, they will cause enough resources to be diverted to trying to make them happy that the computers will no longer run.
I've heard Coopers is a much better brew than Fosters
-Stu
Being a sysadmin for the last 8 years or so, I can sum up what's required for the job in one line:
If no-one knows you exist - you've had a good day.
M.
To make a long story short (and the flame war got ugly), Craig feels that a DNS server needs to support the legacy BIND zone file format. Dan, obviously, does not; he feels that it only matters that one can transfer the zone file over to the new format (losing all comments in the zone file in the process).
Now, I will side with Dan here. Keep in mind that my viewpoint is rather biased, being the person responsible for the MaraDNS server, a server which Craig uses but feels is "poorly written code". Now, the only specific that Craig went in to when pointing out that he did not like my DNS server is that fact that, like Dan's TinyDNS, MaraDNS has no support for BIND's zone file format.
Now, with all due respect for Dan, I think he should not knock a gift horse in the mouth. The fact of the matter is that the code for MaraDNS is open; if support for BIND-style zone files is important to Craig, I suggest that he start coding it himself. I will gladly accept code which can read BIND-style zone files and make it part of MaraDNS.
I am not saying that BIND style zone file support is unimportant. However, I think Craig should be a little more courtious in requesting this feature than badmouthing MaraDNS on the Debian ISP mailing list.
I am sure he is an excellent system administrator; I really wish that he would start up a serious open-source project so that he understands how we OSS coders feel. I think it would make him interact with us in a more mature fashion; and save both him and the developers he flames some grief.
- Sam
P.S. I know Craig already knows this, but there is a non-BIND DNS server which supports BIND style zone files called NSI. It is on the list of DNS servers on my web page.
The secret to enjoying Slashdot is to realize that it should not be taken too seriously.
Yes, very much the system is the users and the structures they function in as well as the computers and the structures they function in.
I remember when I started getting involved in the Net back in the days before AOL was there, and I would describe it to people, and some would talk about being more interested in people than computers, so they didn't want to do the Net. I would tell them that the value of the Net was entirely in the way it enables people to connect themselves with other people to communicate and interact.
The hardware and software are channels, but the content of those channels come from people who contribute it (much/most of it without regard to their ability to comprehend the way the hardware and software enable them to do so).
System admins tends to be control freak. This is not an accusation made by people don't know how to do the job. I was managing computer networks 10 years ago, and understand every bit of the work. I have to say this, system admin is a labor intensive job rather than a tech intensive job. But the environment has changed a lot. The environment is getting better and better, not worse. When I was managing the network ten years ago, I deal only professionals understand computer well. Now, they have to deal with many people not familiar with computers. So, these days, system admins assume they know everything, and others know nothing. This causes the "control freak" symptom. Frankly, the job load of system admin has been reduced greatly compared to ten years ago. I don't know why these guys are whining about.
Me: It burns when I piss.
Doctor: I see. What do you do for a living?
Me: I'm a computer programmer.
Doctor: Really? I have a problem with my email...
You jest? If you said 1970-1973, I could understand your attempt at wit, but 1978? Even Bill Gates had banged out code in 1968.
he had this gal as a coworker yum
like, duh
parent poster is the subject of the article, Craig Sanders