Slashdot Mirror
Nine Traits of the Veteran Network Admin
snydeq writes "Born or made, network admins share certain defining characteristics. Deep End's Paul Venezia offers nine: 'I hope that this insight into the extremely logical, yet consistently dangerous world of the network admin has shed some light on how we work and how we think. I don't expect it to curtail the repeated claims of the network being down, but maybe it's a start. In fact, if you're reading this and you are not a network admin, perhaps you should find the closest one and buy him or her a cup of coffee. They could probably use it.'"
The network is down.
For many reasons there are few things add excitement to life like working with someone who habitually answers the phone with, "I didn't do it."
much of left-wing thought is a kind of playing with fire by people who don't even know that fire is hot - George Orwell
devotion to duty
Religous speak to God. Insane are spoken to by God. When all shut up, one can finally hear Shostakovich in peace
Why do I feel like I just watched somebody jerk off?
<xml><I><am><so><damn>Web 2.0</damn></so></am></I></xml>
This is like a LinkedIn article. 8 ways to do this, 5 systems to improve that, 3 things to avoid when doing the next thing and 9 traits of the successful Whatshisname.
It's maddening.
Humilty
I just bought an MCSE or MSCE or whatever you call it and I was wondering how the fuck I could fool people into believing I actually knew what the fuck I was doing.
Problem solved!
Thanks a lot, InfoWorld!
Captcha: "secured"
...I've never met a network admin who will ever, ever admit that the fault was in their domain. This especially applies to dealing with BT, who once denied that a problem was theirs, and when it mysteriously went away said "oh, yes, a switch was down, but we had a change freeze so couldn't work on it for a few days.". Note the lack of apology for the previous lie.
The guys who write the plot for internet have ran out of ideas.... Just like TV did years ago :(
Nah, it's just another piece of wr1t1ng f0r f00d by someone who does little else.
If that's what you see on linkedin, then you're spending too much time there. It's the same tripe that fills industry rags and like websites like computerworld.com who even translate this sort of thing into many languages to attract yet more little flies into their web.
This guy is what they call an opinion maker. And if this is the sort of opinion he's making, he can be safely ignored. All the more so because the people who would find what he has to say interesting (brass of various stripes) wouldn't know the good stuff when it bit them in the arse and wouldn't understand it either.
That he keeps getting linked on /. perhaps would be maddening if I could still give a flying fsck.
(as mentioned in the article)
1: We already know it's down
2: If we don't know it's down, it's probably not down
3: We will ping and test several times before digging into the problem
4: Believe it or not, we've tried turning it off and back on again
5: During an outage, we're not just staring at the screen -- we're following a path in our heads
6: We calculate subnet masks and CIDR as easily as breathing
7: We do not tolerate bugs; they are of the devil
8: We can read live packet streams and write highly complex filters in our sleep
9: We take big risks all the time
Notice he said "turning an interface on and off", not "rebooting".
Nothing says "I'm a noon and came from a Windows world" like rebooting a switch or router to fix a problem.
Logs on those devices are in memory. Rebooting clears the logs and you then can't troubleshoot. If you can't troubleshoot, you'll never know what really happened. If you don't know the root cause of the failure you can't prevent it from happening again.
Learning HOW to think is more important than learning WHAT to think.
1. A complete disdain or hatred of lusers.
2. A collection of blackmail materials.
3. Homicidal rage.
I am officially gone from
Veteran network admin trait No. 10: We like writing about how awesome we are
Every once in a while we realize we're just glorified factory workers operating machines we know very little about. It is at times like these when we need to point out to ourselves how our brains are amazingly special for being able to navigate a maze of possibilities (completely unlike a delivery boy) and how we can handle the incredibly complex mathematics of subnet masks. It is to remind ourselves that restarting something and waiting for it to fix itself is actually a mental process reserved only for the enlightened few and we must publish such so the world may know of, though not quite comprehend, the extent of our genius.
Slashdot social media options: AIM, ICQ, Yahoo, Jabber and Mobile Text. Why no MySpace?
"Veteran network admin trait No. 1: We already know it's down Few things are more annoying than having your phone blow up with automated alert messages from your monitoring systems, scrambling to dig into the issue, only to be continually bombarded with humans texting/talking/emailing/calling with the same "Is x down?" question, or even worse, "The network's down!" If the outage is significant, we already know about it, and we are trying to work on it as fast as we possibly can. Continued attempts to deliver elderly information will only impede that effort."
Said as an admin which only looks at its own patch of land. The reality of production, is that you have router, network, firewalls, backends server, front ends server, and so forth. Pray tell us how people know its is your network and not many of the other point of failure ? People asking "is x down" are not a symptom of elderly info, it is a symptom y
"Veteran network admin trait No. 2: If we don't know it's down, it's probably not down Conversely, if we get a message claiming, "The network's down!" yet we have not been notified by any monitoring system, then the problem is almost certainly the complaining user in question." yeah right, because all those system are quasi instant and you are not distracted by something else, as a perfect human. "No, it isn't -- reboot your laptop." I see. You are an idiot.
"If we begin looking into a problem, especially across a WAN or long-haul link with several providers in the middle, we will reserve judgment for the first several minutes. This is because these connections are subject to the vagaries of their path, and connectivity problems can come and go like ghosts in the night. A fiber WAN link that was stable a minute ago but is now exhibiting 30 percent packet loss will more than likely fix itself in short order." I will tell you the same you told the poor user previously : "reboot your router". If not reboot your brain. A network which exhibit such behavior has problem.
"Veteran network admin trait No. 5: During an outage, we're not just staring at the screen -- we're following a path in our heads" nope you jsut said before : first you wait time to wait that it fixes itself in short order. You damn said it yourself.
"Veteran network admin trait No. 9: We take big risks all the time Network admins tend to work on many remote devices. Unlike server admins who can pull up a console if the server is otherwise inaccessible via the network, we usually have no such luxury. This means that changes we make to certain devices carry with them the ever-present threat of the loss of connection. Basically, we're always a missed keystroke or two away from making a problem worse, or causing a big problem where none previously existed. " I have always been one typo away from destroying a whole system. Read : destroy and need very long (8+hours) backup to restore, not make temporary inaccessible. You think you are so special snowflake ? We backend admin can be a keystroke away to *destroy* a whole DB inadvertently. Go cry me a river.
Sorry but the whole reek of "i am so special and important". You are not. You are a cog. In fact, as a "plumber", you might make temporary mistake, but you are nowhere near as dangerous as backend engineeer and administrator.
so, you are just doing your job and would like some free coffee for it?
be happy with your paycheck like everybody else who is doing their job every day, you're not special....
Let's not forget that antisocial trait where we frown on other lesser knowledgeable mortals as we stroke our massive long white beards.
A 'singular oddity' is an event that cannot be explained and only happens when you are alone.
Cooperative and uncooperative are the two primary traits of network admins. Seeing that most network admins are ignored when everything is going well and cursed when things are going badly it quickly becomes obvious to most network admins that saying no to most requests becomes a survival trait. But this often means that a network admin starts actively interfering with the actual business (unless the business is network administration)
So take your average non IT company. The sales people want to switch from Blackberries to iPhones and Androids. The network admin has 4 certifications from RIM and recently set up a huge Blackberry server. Plus this particular admin hates all things touch screen. The usual scenario will be that the admin will pull out 9 whitepapers showing just how secure Blackberry is as compared to iOS and Android. Using techno babble he will explain how hackers will be running the company in under a week. The salesman on the otherhand say that they look like tools from the 90s with their blackberries. The network admin wins the battle but then he starts to get nervous as he notices traffic on his BB server is nearly zero. A quick investigation shows that nearly every Blackberry sits in a drawer and the Salesman have gone out and bought their own phones and swapped the SIM cards. The network admin sends out a memo saying this is against corporate policy which is ignored. His attempts to get a salesman fired(to set an example) for violating security fail. He then notices nearly everyone is using gmail instead of his highly secure MailMaster2000. Then sends out a memo indicating that this is against corporate policy. He then implements a 30 day mandatory password rotation. Internal file server traffic nearly drops to zero because everyone switched to dropbox. He then sends out a memo that dropbox is against corporate policy. He then starts blocking sites such as reddit and he notices that network traffic drops to nearly zero. But walking by a sales person's office he notices that they are on reddit. So he investigates and finds out the entire sales team has bought mobile data plans. He then sends out a memo saying that private data plans are against corporate policy.
Then he comes to work only to find a contractor in his office. The contractor is there to "rationalize" IT seeing that after the IT guy insisted that all apps be developed for BB first the sales people have gone out on their own and developed 3 smartphone applications that have increased sales by 80% and that promotions via Reddit have sent corporate website visits through the roof. The company now works with clients via dropbox much more successfully than with the sftp system that merely served to confuse before. With mobile dataplans the salesforce has become much more effective.
Now the IT guy is left filling out a resume where his two best features are many Novell certifications and many Blackberry certifications.
IT people shouldn't cave into every whim of the week but I have seen so many that are stuck in the thinking of whatever year they became head of IT. IT is just one tiny department in so many companies yet I have seen IT somehow be able to treat senior managers of other departments like children. Seeing that they aren't children they often discover the virtues of outsourcing. The key benefit of outsourcing being that if the people they outsource to try pulling any crap they can be dropped in a second.
The author is trying to make a big deal about how great the network admins are. Can't blame him for feeling like working in a chain gang from some Alabama prison, deployed on contract in a quarry, with the tool chest consisting of just one sledge hammer. Looks like most of the time they ask the user to reboot their machines (trait 2) or reboot network switches (trait 4) or wait for the problem to solve itself (trait 3). Other traits seem to be putting on a brave face, telling themselves how smart they are. I am sure you could find nine such traits for plumbers clearing blocked sewers too, except they can't reboot the sewers nor wait for it to unblock itself.
sed -e 's/Chuck Norris/Rajnikant/g' joke > fact
I did of course phrase the subject to maximize the provocation :-) but my point is, while I really like the plumping to work, I don't really care about "the nine traits of a veteran plumber". For me a network admin is (believe it or not) a LOW TECH job, using of-the-shelf standard components (as any other admin/technician/mechanic). If my mechanic has to stare blankly while mentally figuring out what is preventing my car to start, I will let him do that in peace - because I assume he knows better than me how to do his job.
I have NO doubt that network administration is an intriguing job, with its share of problems to solve. But as long as the down time is counted in hours and not days, I prefer them to work invisible behind the scenes, not bothering me and hindering me to do my job.
Don't get me wrong, I know that a good admin is well worth his pay. But also I love the cleaning personal, especially when they don't clean while I'm working :-)
I guess I was hoping to gain a little insight in how the admin job is done - I dunno, maybe adapt some of it to my little (tiny) home network. But most of the points was fairly obvious - yes I also reboot my router. No of course I can't calculate subnet masks/read packages/write filters in my head. Is it impressive he can? Slightly, but not much, skills like that develop by regular use. I would be more surprised if he had written "we just google it" :-)
Slashdot "articles" have finished scraping the bottom of that barrel and have commenced drilling operations. What a joke.
I left the world of systems administration and network management in 2006 when I took a position as a software engineer... I've kept my hands wet with managing my personal web servers and those of a few hosting customers, but I do admit to missing those sysadmin/network admin days.
The statement "use it or lose it" really is true - I still have all the troubleshooting skills and nowadays, a LOT more insight into the software side of things, but I'm sure I'm rusty as hell with IOS and such. It's also a significantly more hairy security landscape today than back then, so I guess I can reminisce about "the good old days" from the safety of my bunker - er, home office.
The Digital Sorceress
If you are good, you know when the network "isn't right" which is much better than UP/Down.
This includes knowing things like a switch port has lost duplex on an ethernet connection,
there is a broadcast storm in a building/site, one of your redundant links is down.
Being right much much more often than being wrong in diagnosis/troubleshooting is also a good trait.
Goes for System Admins too: Used to commonly having to work all night to do routine maintenance then come in the next day. While the rest of IT/Development is sleeping these guys are up all night. There isn't much glamour being on conference calls at 3 AM.
I am a DBA and I generally like to be certain I need to contact an SA or Network Administrator before I do, given the insane hours these guys work. My ultimate fear is that they will quit then Ill be forced to do their job. So I need to be nice to them.
Note: #2 requires systems access to the email server (or spam filters) as well as fileshares - something a pure network admin wouldn't have.
Quo usque tandem abutere, Nimbus, patientia nostra?
95% is comprised of defending the network. If it's been up and running, no changes have been made and no equipment has died... chances are, it's NOT the network. Go check group policy.
"Please, shut up. Just when I think you can't say anything more stupid, you speak again." -Archie Bunker.
No it doesn't: packet sniffers.
I am officially gone from
insufferable prima donna
Every job that you make a living can (and perhaps should) be considered glamorous -- even if it just sweeping the streets or emptying the trash bins.
I love hearing different professionals describing their professions in detail; describing the small things that can go wrong and the oddballs cases they have encountered.
I perked up because I thought it had something to do with former military..
The article's author uses the "we" a lot, which strikes me as awkward that he's including himself in the group of "Network Admin" when he writes things like;
Veteran network admin trait No. 1: We already know it's down
Veteran network admin trait No. 3: We will ping and test several times before digging into the problem
Veteran network admin trait No. 5: During an outage, we're not just staring at the screen -- we're following a path in our heads
Veteran network admin trait No. 6: We calculate subnet masks and CIDR as easily as breathing
I guess the term "veteran" is subjective. He should add a new trait;
Veteran network admin trait No. 10: We write articles like we're fulfilling a quota
SNNNOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOORE.
And the number one reason to not reboot if not absolutely necessary: you never know if the system will actually come back up.
Bullshit. Well, either that or our network admins are numpties. We had a switch go down in a data hall. We knew the switch was fubar as we'd lost connectivity to a number of devices at the same time. Networks wouldn't admit a fault. We eventually got moved to new switch ports and lo, everything started working.
The usual approach to any network fault in our place is "replace your NIC". That has, in my memory, fixed maybe two faults.
We're supposed to have a network admin, he is never around, never fixes anything that is broken, and closes tickets after sitting on them for a week with no resolution or fix listed in the summary. We currently have a situation where we lose all connection to the data center at lunch time, been this way for 5 weeks now with 2 tickets closed as unreproduceable because the network admin tried accessing the servers at the data center later in the afternoon. He is almost as bad as a call center script reader in India, if we send in a detailed ticket with all the troubleshooting and information listed the first thing he does is ask us to try pinging the server which was already covered in the information given. Useless pile of shit for a human being making the lives of the real heros (desktop support) a stressful nightmare.
Top ten things that are shit:
1) Lists of ten things that are shit.
2) Lists of approximately ten things that are shit.
3) Everything else.
Confucius say, "Find worm in apple - bad. Find half a worm - worse."
are you sure it's not the network? True story: A long time ago, I was a server admin for a very large oil company. I was handed a trouble ticket saying a print server was down. I quickly checked, saw that the server was up, I could access it, people were printing from it. I handed it off to both the deskside support person, and the network admin, and said server is working, either the user messed up their network connection, or the network is having a problem. I copy the help desk. Two more "the server is down" tickets come in, and in the meantime, the network admin rejects the first ticket saying the network can't be down. Over the next few hours, this escalates, as does the finger pointing. I say, ok, let's have a server guy, network guy and desktop guy meet on that floor where all of the problems are happening, and we will get it fixed. Now I should mention that this was on Token Ring, you whippersnappers should look it up on wikipedia, and while you're there, you may as well Google "whippersnapper". All of the tickets were coming from one half of one floor of the building, nowhere else in the building. Still, I get to the wiring closet, where I see a clueless desktop guy (sent by his manager) and a very, very angry and obstinate network guy (no doubt, sent by god). Network admin still insists that it can't be the network. I hear a noise, it is a persistent, steady click, click, clicking. I know what the noise is, the network admin is clearly deaf as well as stubborn. I say, "what is that clicking noise?" He stops and listens finally, and gets a deflated look. He knows what the noise is, and even worse, he knows that I know what it is. I don't gloat, I just look at him, and politely say, "you will fix it, and then close the ticket when you're done, right?" He can't even look me in the eye, but ignores me and stumbles into the jungle of his wiring closet looking for his broken network hub.