Nine Traits of the Veteran Network Admin

snydeq writes "Born or made, network admins share certain defining characteristics. Deep End's Paul Venezia offers nine: 'I hope that this insight into the extremely logical, yet consistently dangerous world of the network admin has shed some light on how we work and how we think. I don't expect it to curtail the repeated claims of the network being down, but maybe it's a start. In fact, if you're reading this and you are not a network admin, perhaps you should find the closest one and buy him or her a cup of coffee. They could probably use it.'"

Comments

I RTFA

[19thNervousBreakdown]

Why do I feel like I just watched somebody jerk off?

Re:I RTFA

[Anonymous Coward]

Because you can't handle the sheer awesomeness that is the veteran network admin!

Re:I RTFA

[evilviper]

They mostly come across as a bit OpenBSD-mailing-list if you see what I mean. They are defensive as hell, maybe their jobs make them that way.

Imagine running your company's entire IT infrastructure off of a random assortment of Android cell phones... That's basically the job of a Network Administrator.

Despite running the most critical piece of infrastructure, they can't peer into the base system at all, to see how it's going to behave in edge cases. The monitoring and management tools available are shockingly poor, and offer little utility.

They're forced to use a random mix of different network hardware, which all behaves differently, and each has a list of errata 100 pages long. Updates are huge monolithic firmware blobs, guaranteed to have bugs, leading you to choose between dealing with the awful but barely manageable bugs you have now, or updating and dealing with a whole new set of bugs, and potentially some so bad they'll take the system down, and there's relatively little you can do to test this, even if you have a rather large testing network (which most do not).

Software Defined Networking and OpenFlow look like they might finally start changing this, and Network Admins will be able to understand how these complex systems work, and even define the behavior they want. But in the mean time, your Network Admin remains the Prime Minister of the insane asylum.

Clarifying #4

[chill]

Notice he said "turning an interface on and off", not "rebooting".

Nothing says "I'm a noon and came from a Windows world" like rebooting a switch or router to fix a problem.

Logs on those devices are in memory. Rebooting clears the logs and you then can't troubleshoot. If you can't troubleshoot, you'll never know what really happened. If you don't know the root cause of the failure you can't prevent it from happening again.

Re:Clarifying #4

[mysidia]

Logs on those devices are in memory

Although they should be on the NMS and syslog servers too... and many Juniper M/T series devices, which are very frequently used by large carriers for core and edge routers, have a hard drive mounted on the /var filesystem, that the routers' syslogs get written to.

Rebooting sounds like an act of desperation. It's almost never the cleanest way, and it almost always incurs additional downtime, causes more outages, or further lengthens the network downtime --- since you require 3 to 5 minutes for a reboot, then maybe 3 to 5 more minutes for routing protocols to reconverge.

And god help your soul, if you reboot a Cisco device such as a 72xx or Catalyst 65xx running BGP with 3 or 4 copies of a full table, and with 8 or 9 total peers.

That one weak moment, when reboot was chosen may result in 60 to 90 minutes of trying to coax the network back up gently.

Re:Clarifying #4

[NatasRevol]

If you don't have a separate syslogd system, you're not doing your job well.

Veteran network admin trait No. 10

[mwvdlee]

Veteran network admin trait No. 10: We like writing about how awesome we are
Every once in a while we realize we're just glorified factory workers operating machines we know very little about. It is at times like these when we need to point out to ourselves how our brains are amazingly special for being able to navigate a maze of possibilities (completely unlike a delivery boy) and how we can handle the incredibly complex mathematics of subnet masks. It is to remind ourselves that restarting something and waiting for it to fix itself is actually a mental process reserved only for the enlightened few and we must publish such so the world may know of, though not quite comprehend, the extent of our genius.

Re: Will read later

[nosfucious]

These "traits of" or "habits of" articles are the modern equivalent of horoscopes. Lots of feel good stuff, but not much actual advice. I can agree and disagree with every point to some extent.

Nice article that generates a lot of page views. For real advice get 1 or more O'Reilly books, or better yet, find a competent admin and become his/her apprentice.

Trying to add galmor to regular job.

[140Mandak262Jamuna]

The author is trying to make a big deal about how great the network admins are. Can't blame him for feeling like working in a chain gang from some Alabama prison, deployed on contract in a quarry, with the tool chest consisting of just one sledge hammer. Looks like most of the time they ask the user to reboot their machines (trait 2) or reboot network switches (trait 4) or wait for the problem to solve itself (trait 3). Other traits seem to be putting on a brave face, telling themselves how smart they are. I am sure you could find nine such traits for plumbers clearing blocked sewers too, except they can't reboot the sewers nor wait for it to unblock itself.

Re:Obligatory xkcd

[Anonymous Coward]

Speaking of lumps of useless meat, sys admins are just CS dropouts YO!