Slashdot Mirror
Documenting a Network?
Philip writes "Three years ago I was appointed as a network manager to a barely functioning MS-based network. Since then I've managed to get it up and running — even thriving — but have been guilty of being too busy with the doing of it to document the changes and systems that were put in place. Now as I look back, I'm worried that I am the only one who will ever know how this network works. If I get hit by a bus or throw in the towel for any reason, I'd be leaving behind a network that requires some significant expertise to run. Ultimately, this won't be a good reference for me if they are trying to work out technical details for years to come. It looks like I'm going to have to document the network with all sorts of details that outside consultants could understand too (no, I don't want to be the outside consultant), especially since it's likely that my replacement will have less technical expertise (read 'cheaper'). Are there any good templates out there for documenting networks? Is anyone who has done it before willing to share some experiences? What did you wish your predecessor had written down about a network that you inherited?"
Sounds like a very easy way to over work and over stress your self, get some help one way or another. Summer is coming and I'm sure there are plenty of Comp Sci/Network Engineer/IT students that could of help. It may not be a bad idea if you make a plan of some kind before you go head in.
The next step up from a Post-It, though, is a snazzy label-maker. My portion of the company uses these extensively to document our development lab (we do some NMSy stuff). Of course, it's not a production network, and standards are a little different.
The World Wide Web is dying. Soon, we shall have only the Internet.
Whether the comment was intended to be funny, I find this to actually be a serious issue...
The phone numbers/emails for points of contact in other departments/companies.
You likely don't run *everything* and the new person needs to know who to contact when the interaction between inside and outside fails.
20 characters max for the password? How will I use my favorite poems as passwords?
Draw a horrible diagram in Visio (or similar) of what's connected where without any indication how it actually works!
Not really, because as most high level executives know, IT doesn't really do anything!
Mod me down, my New Earth Global Warmingist friends!
...or if you show it to them they won't bother with reading it
This is more to the point. Most network admins have the attention span of a flea and won't read past the first sentence of anything you write; actually, I could probably expand that statement to include most people generally. Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.
"The network started horrible, here is how I cleaned it up" is a GOOD reference. I have killer references from two jobs I automated myself out of this way. Each time I got a more interesting more challenging better paying job by doing so.
Present client I am at I inherited a network of about 15,000 clients that was previously managed my a very incompetent IT department. Started by looking at the existing flowcharts and discovered that almost everything that was documented was wrong... Long story short I have been spending a fair bit of time reverse engineering their production environment so that we could accurately document it. Unfortunately we had come to the conclusion that we can use /nothing/ that the previous administrators left behind for documentation. You don't want someone like me coming in and looking at your documentation and declaring you incompetent, it can cost you your job.
You haven't detailed the size of your organization to know if you will need sign off from other departments or not. If possible try to get sign off so that they have a reference and you can create a standard that can be used to fix things and to ensure your designs don't get trampled by a new admin in another department. You really need to provide more detail on your environment for people to answer you.
I do most work in Visio, starting at 50,000 feet and working my way down. At this level I need to document network topology, server distribution and database server distribution. I work my way down from there using a zoom in style that has served me well for 30 some clients. Depending on the size, complexity and your area of responsibility you may need to flowchart anywhere from a 2-3 levels to potentially dozens of disparate processes. You haven't mentioned much about process development, I assume you want people to know how to do at least critical portions. Never write a process without flowcharting it, this will save you grief by getting people to focus on the process instead of a step by step set of directions. It takes someone fairly good to document the complex and make it look simple, that is your job at this point in time.
The bottom line is that your documentation should show dataflow for each critical system. As long as you can do this someone else can step in and work with what you have, even if they may not understand a given piece. One of the big advantages of flowcharting everything (especially processes!) is that this will readily show you weakness and holes that may have been previously overlooked. When flowcharting complex processes don't be afraid to have a single point represent an entire additional complex process that can be distictly referenced of it's own accord (as an car repair manual of mine once described the process to replace a crankshaft "Step 1. Remove Engine".) If you try to put to much detail in a given process you lose your audience and the value of the documentation.
Bottom line when I am done with a design document it covers server, network, database and client topology in varying levels of detail with dataflow. A typical design document I would turn over would be 150 pages with most of that broken down into different sections describing what was done, why it was done, the best practices followed for build, and best practices for lifecycle. The document typically does not get read by any one person, instead it would be a reference for a number of different departments that will each reference it according to their own needs.
If you are in the server room, and you have:
A: a spreadsheet that your predecessor made.
B: a post-it note on the switch saying it what it does.
Which one do you trust?
For the physical/low-level network the documentation should be in the network. Just like source code should contain comments about this particular piece of code, a similar approach works reasonably for the physical network. I see no point in a having an outdated spreadsheet. It is more useful that the cables and ports are labelled and numbered, that there is a post-it note on a switch say where the links go, etc.
The grand overview should be in electronic form, though. A scanned hand drawing is fine. A photo of a whiteboard drawing is fine too.
For the logical network put comments whereever possible. On settings, VLAN configurations, server connections, account setups, ...
Again, the grand overview should be in electronic form.
I have found it useful that the information is timestamped, so you know when it was last valid.
A summary diagram (dia/visio) is still pretty key, but nmap (or specifically the latest versions of zenmap (the gui frontent to nmap) provides a good detailed view of network topology and if you include the capture file, you can annotate specific computer details in the notes section (ofc you have to assume your replacement will be familiar with zenmap for that to be useful though)
IranAir Flight 655 never forget!
No wonder our field and many of our professions have such a bad reputation.
I have read only a few posts and two (moded up 5) say pretty much to ignore the issue.
In several networks I have worked with fundamental information was non existent. This translated in lost time, down time and actually losing money (if you lost your job in one of those companies recently, the indolent SAs or Network administrators may be partly to blame).
You never know who the next guy will be, if he is less experienced or capable then the documentation will be very valuable, if he is more experienced or capable then you would have saved their time to do some real work, after all they (and you) have not being hired to do forensics.
How a professional can hide behind the "let's be real" nonsense is beyond the pale.
IANAL but write like a drunk one.
Attempting (even facetiously) to blame SPAM on Windows is wrong. If every copy of Windows on the Internet somehow magically disappeared, the SPAM problem would not abate. Bot herders and spammers would simply shift their efforts to other platforms.
If your doubt this, consider what the winner of this year's PWN2OWN contest had to say about why it's easier to target Mac OS X.
BTW, this is not a troll, and I'm not a (Windows|Mac|Linux) evangelist of any kind. I just find kneejerk Windows bashing rather tiresome
That's always.
Those who don't document don't have job security. They are an insecure leach sucking up a paycheck fearing -- and rightfully so -- the day they are going to be replace.
Those who DO document show their value to the organization, and should have no fear of being replaced. Their position is secure -- and should they go elsewhere -- they have something to show of and for their work.
I disagree with the parent vehemently and will say so based on years of experience as a techie, a techie manager, a manager of techies, an executive, and (thankfully) a techie again. You can never document too much, but those who don't cost the organization more in the long run each and every time.
Document. Document well and often. Ignore the parent.
Ehud
At which point you won't be getting the passwords anyway since you fired him/her? Good idea. ^_^
You can't take the sky from me.
So what happens if said predecessor gets hit by a bus, has a heart attack or a stroke and can no longer tell you the passwords? Or, worst case scenario, the whole IT team gets taken out in a road accident on the way to a team building session for example? I've read a few "deserves to be fired" comments on /. and usually tend to agree (or occasionally get embarrassed because I think hmm, that's me!), but in this case you are being a fool.
Of course, if you are dead then you won't care if they have the passwords, but some of us actually like our places of work and even our colleagues, and want our place of employment to be able to chug along even in our absence.
which is totally what she said
If the predecessor does write the passwords down, he deserves to be fired.
That's knee-jerk stupidity, and you should be ashamed of your non-thinking fundamentalism.
Passwords do need to be written down, and stored in "escrow". I put the list of passwords in an envelope, lick seal it, sign and date the seam, and then seal it again with clear packing tape. Give it to the boss to put in his safe.
Yes, it's easy to open, but you'd know whether someone tried to tamper with it.
"I don't know, therefore Aliens" Wafflebox1
circuit ID ? in that a farmer's synonym for "MAC address" ?
Not everything is ethernet.
Yay me!
My school's network admin used to say that when he didn't have to do anything at all during a work day, he completely deserved his pay.
The Wise adapts himself to the world. The Fool adapts the world to himself. Therefore, all progress depends on the Fool.
Either he writes the passwords down, or he uses weak passwords that a human mind can remember.
Besides, a password is a security token. A piece of paper or a little plastic card with the password printed on it or a USB stick with SSH key or whatever saved on it are also security tokens. They aren't inherently less secure than memorized passwords; you simply have to secure the physical object by, for example, locking it in a safe.
"Passwords should never be written down" is an idiotic rule, right there with "never use goto".
Forget magic. Any technology distinguishable from divine power is insufficiently advanced.
Yeah, I think that's more or less true. At one of my previous jobs, I had a guy try to imply that I didn't deserve my pay because I "wasn't doing much". When I asked him what I should be doing, he said, "It's just that you have a really easy job. The IT guy at my last job had it much harder. He was always running around, fixing things. You just sit at your desk because nothing ever breaks."
I can't remember now, but I think I might have done a literal facepalm right then. I said something like, "Has it occurred to you that, if you think none of our IT stuff ever breaks, I must be doing a good job? If the IT stuff at your last job kept breaking all the time, he was doing a worse job than me?"
if they can't replace you they can't promote you
I also try to stay away from documenting things in static formats. If you ask me, incorrect documentation is even worse than no documentation at all, and the fastest way to get incorrect documentation is to create a process that relies on a person doing all of the updating manually.
I know a lot of people love their spreadsheets and their diagrams, and maybe they update them religiously. Nonetheless, that process is *always* prone to error. And if a technician goes to a document for information and finds out that the document is wrong, the document loses its credibility. If that happens a few times, the technician will simply stop trusting the document, and it will just fade into obscurity.
If you want to document a system, look for ways to make the system document itself. Switches keep real-time lists of the MACs that are connected to them. Routers keep real-time lists of which MACs map to which IP addresses. Routers and switches will always tell you their current configuration if you ask, and you can automate the process of asking and storing and checking for changes. Most servers will tell you their serial numbers automatically if you ask them, so you should automate the process of asking them and storing that information. The same goes for what kind of hardware is in the server, where the server is attached to the network, etc.
So much information can be collected automatically rather than recorded by hand, and when you collect the information automatically, it will always be up to date. It will not matter if a tech decided to re-rack a server in the wrong place -- even if they didn't write it down, your network knows that it moved, and it will tell you if you ask it. So the next time you sit down to write a 200 page document describing the network, you should ask yourself a) how much will it cost in time and effort to keep this document relevant, b) how likely is it that the document will become out of date either through accident or negligence, c) how quickly will people abandon this document if it does become out of date, and d) aren't there huge parts of this document that could be totally dynamic instead of written in static text on a page?
Something I found to be VERY good policy that was implemented at my former position (as Network Admin) was to hand to the boss (CEO, CIO or whatever) a sealed envelope with EVERY relevant password (most importantly the admin password :) ), to be held in the company's vaults (if your company has such a thing of course, or similar).
:)
Whenever an important password was changed, I would hand over the new envelope
There are three kinds of lies: lies, damned lies, and statistics.
I was taught that the proper way to store admin passwords for an emergency situation (such as admin gets hit by bus) was to
1. Write each password on an individual piece of paper
2. Seal each piece of paper in an envelope
3. Store the envelopes in a Safe/Deposit box that a limited number of people have access to, (company owner, CTO, CEO, IT Manager)
4. Put policy in place that requires passwords to be changed and re-recorded any time the envelope it is stored in is opened.
It may seem low tech, but it is probably the best solution for a small to medium sized business.
That's my USD$0.02
Didn't Bruce Schneier once say "If you think cryptography can solve your problem, then you don't understand your problem and you don't understand cryptography"?
I don't think this solution is nearly as good as the envelope system. In particular, I don't expect the company owner or the CEO of the average small to medium business to have the expertise keep their private key files and password accessible to them and no one else. That means that either you're actually relying on two out of the other three keys being available (if those two fail to keep their private key files accessible to themselves) or you have poor security (if those two fail to keep their private key files accessible to no one else), whichever is worse.
In contrast, all of the trusted people probably know how to keep a physical key reasonably secure and accessible, and if you use a deposit box at a bank there may be identity checks as well.
In general, if your solution requires detailed technical knowledge from non-technical people, your solution is broken. Pick something low-tech instead.