Nine Traits of the Veteran Network Admin

snydeq writes "Born or made, network admins share certain defining characteristics. Deep End's Paul Venezia offers nine: 'I hope that this insight into the extremely logical, yet consistently dangerous world of the network admin has shed some light on how we work and how we think. I don't expect it to curtail the repeated claims of the network being down, but maybe it's a start. In fact, if you're reading this and you are not a network admin, perhaps you should find the closest one and buy him or her a cup of coffee. They could probably use it.'"

Comments

I RTFA

[19thNervousBreakdown]

Why do I feel like I just watched somebody jerk off?

Re:I RTFA

[Anonymous Coward]

Because you can't handle the sheer awesomeness that is the veteran network admin!

Re:I RTFA

[evilviper]

They mostly come across as a bit OpenBSD-mailing-list if you see what I mean. They are defensive as hell, maybe their jobs make them that way.

Imagine running your company's entire IT infrastructure off of a random assortment of Android cell phones... That's basically the job of a Network Administrator.

Despite running the most critical piece of infrastructure, they can't peer into the base system at all, to see how it's going to behave in edge cases. The monitoring and management tools available are shockingly poor, and offer little utility.

They're forced to use a random mix of different network hardware, which all behaves differently, and each has a list of errata 100 pages long. Updates are huge monolithic firmware blobs, guaranteed to have bugs, leading you to choose between dealing with the awful but barely manageable bugs you have now, or updating and dealing with a whole new set of bugs, and potentially some so bad they'll take the system down, and there's relatively little you can do to test this, even if you have a rather large testing network (which most do not).

Software Defined Networking and OpenFlow look like they might finally start changing this, and Network Admins will be able to understand how these complex systems work, and even define the behavior they want. But in the mean time, your Network Admin remains the Prime Minister of the insane asylum.

Clarifying #4

[chill]

Notice he said "turning an interface on and off", not "rebooting".

Nothing says "I'm a noon and came from a Windows world" like rebooting a switch or router to fix a problem.

Logs on those devices are in memory. Rebooting clears the logs and you then can't troubleshoot. If you can't troubleshoot, you'll never know what really happened. If you don't know the root cause of the failure you can't prevent it from happening again.

Re:Clarifying #4

[mysidia]

Logs on those devices are in memory

Although they should be on the NMS and syslog servers too... and many Juniper M/T series devices, which are very frequently used by large carriers for core and edge routers, have a hard drive mounted on the /var filesystem, that the routers' syslogs get written to.

Rebooting sounds like an act of desperation. It's almost never the cleanest way, and it almost always incurs additional downtime, causes more outages, or further lengthens the network downtime --- since you require 3 to 5 minutes for a reboot, then maybe 3 to 5 more minutes for routing protocols to reconverge.

And god help your soul, if you reboot a Cisco device such as a 72xx or Catalyst 65xx running BGP with 3 or 4 copies of a full table, and with 8 or 9 total peers.

That one weak moment, when reboot was chosen may result in 60 to 90 minutes of trying to coax the network back up gently.

Re:Clarifying #4

[NatasRevol]

If you don't have a separate syslogd system, you're not doing your job well.

Re: Will read later

[nosfucious]

These "traits of" or "habits of" articles are the modern equivalent of horoscopes. Lots of feel good stuff, but not much actual advice. I can agree and disagree with every point to some extent.

Nice article that generates a lot of page views. For real advice get 1 or more O'Reilly books, or better yet, find a competent admin and become his/her apprentice.