Slashdot Mirror
Documenting a Network?
Philip writes "Three years ago I was appointed as a network manager to a barely functioning MS-based network. Since then I've managed to get it up and running — even thriving — but have been guilty of being too busy with the doing of it to document the changes and systems that were put in place. Now as I look back, I'm worried that I am the only one who will ever know how this network works. If I get hit by a bus or throw in the towel for any reason, I'd be leaving behind a network that requires some significant expertise to run. Ultimately, this won't be a good reference for me if they are trying to work out technical details for years to come. It looks like I'm going to have to document the network with all sorts of details that outside consultants could understand too (no, I don't want to be the outside consultant), especially since it's likely that my replacement will have less technical expertise (read 'cheaper'). Are there any good templates out there for documenting networks? Is anyone who has done it before willing to share some experiences? What did you wish your predecessor had written down about a network that you inherited?"
A good tool like dia which can allow you to create a network diagram. When it comes to documenting a network, a picture can be worth a thousand words. Or you could also use MS Visio as it is, perish the thought, a good tool. A good, detailed diagram can come in very handy as a reference tool for your own use in case of a failure.
Basic network documentation:
I've found that starting out with the very basic physical layout and working your way up in complexity is greatly beneficial.
i.e. start out documenting network cable runs including cable type. follow it by switch layout. follow that by routers and vlan setups. follow that by the servers that provide basic network functionality(e.g. DHCP, etc...). If this is a windows network, that would likely mean detailing the domain controller setups. From their systematically document the systems in order of importance to the business, etc...
Also, visual diagrams are extremely helpful.
For me, visio's are great and everything, passwords too, but really the most valuable thing you can do is document single points of failure, outdated software/hardware, etc., license keys/expiration dates, cert expiration dates, personal support contacts you have and all vendor relationship details as well are essential. Do you use change control? If you do, go back and comment your changes, if not, do the best you can at explaining why things are the way they are. Get some open source software that is good at indexing data and create a searchable knowledge base from the information above. Don't concentrate on docs that can be found on the web at first because any admin worth their salt will know where to look for how to's, etc. Focus on the why's, the where's and disaster recovery.
My two cents...
1. Viseo overview of the network drawing with complex areas drawn out specific detailed viseo's (even a scanned sketch or paint drawing is better than nothing)
2. A spreadsheet with circuit ID's mapped to router and interfaces.
3. Document the trunk interfaces as well as the LAG's (Link Agrregation Groups, port channels, whatever you want to call it)
4. TACACS passwords / domain logins in a secure location (or radius or diameter or whatever you use)
5. Data center capacity as a function of 1. Rack Space, Cooling Capacity, Electrical Load.
6. Write brief knowledge articles describing any problem areas and explaining a history of anything you think would be hard to figure out easily. No need to go hog wild, just re-brand the RCA documentation you have. You do have Root Cause Analysis right?
7. Network protocol hierarchy map. Where are your major redistribution points, what is your routing strategy etc.
8. If you have a voice network document all your DID's, PRI trunks, Gatekeepers, Dial Plan, and any translations you use on h.323 gateways or how MGCP or SIP is configured. If you have a complex call center you should probably pay to have it professionally documented in down to the minute detail.
9. SSID's and BSSID's for any wireless you may have as well as passwords, 802.1x authentication methods, along with linking documentation.
10. Make the documentation part of your CMR process (Change Management Review) and incorporate it into the time allotted for a change.
I know these are just rough ideas and you should get many more ideas from all the smarter people on here than myself, but whatever advice you get I would say you would need to have the documentation update able via subversion, or some document control system and have some kind of review process for it, even if it means getting together over pizza with some of the other groups and asking them about their environment and getting pointers and possibly help on documenting it all. Documentation is a full time effort and IMHO there is no such thing as too much documentation. You would be surprised how good documentation can aid you in problem resolution down the road or aid vendor support in helping you resolve a major outage. The three basic principles of network care are document document document. :)
Cheers,
Anonymous Coward.
On the side, I manage a small network, and I've also wondered the same sort of thing: if someone else needed to find their way around, where would they start.
A Wiki makes for a really nice way to document things, not least because you can include all sorts of cross references. For example, a list of servers, with links to the services they provide - and a list of services, with links to the servers. But Wiki's normally run on servers, which leaves your successor with a chicken-and-egg problem.
A bit of random surfing turned up TiddlyWiki, which is a Wiki in a single HTML file. A really elegant bit of engineering, and very handy for self-contained documentation. Since the entire Wiki is just a single file, it's easy to protect. I wound up with two: one with "public" information describing the general architecture and one with private information (including passwords). The private one you can put on a USB-stick in a safe, hand to your boss, or whatever seems appropriate...
Enjoy life! This is not a dress rehearsal.
Why not use an automated too?
www.open-audit.org
My job requires me to do exactly what you're looking to do but for multiple companies/networks. Then, as soon as I'm done, I usually pack up and go or get hired in and fix the network.
Since I'm writing the Network Overview for managers AND potential future network managers I tend to write mine in the following format:
1) Synopsis of what the network does for the company, what general technologies they use (Windows AD vs *nix OD, thin clients vs Windows boxes, Cisco vs Brand X), and what the LOB software is.
2) Points of contact for the ISP and other providers (anti-spam, anti-virus, hardware, etc). Passwords for various accounts and services.
3) Logical network overview map (visio), containing firewalls, routers, switches, other devices, open/forwarded ports, IPs, what the servers do, what vlans are in place, Quick explainations for why (such as why vlan vs a seperate subnet).
4) Physical map of devices if the complexity of the network calls for it.
5) Software notes, what apps are critical for the business and which systems they rely on.
Then, for my specific job I have to do the following:
6) Licensing issues.
7) Network weaknesses/points of failure.
9) Other rec's.
I have a nicely formatted template page with all those categories set out. I also maintain a page of IP address assignments and an inventory of harware specs of all the machines in the office (which is helpful in the cases of "We need to reproduce a bug that only happens on ____ processor with ____ video card" and of "We're getting new machines. Who is in most dire need of an upgrade?").
I write down everything in these, and find myself referring to them very often. My predecessor gave me a Word document with all his notes in it, which has been very useful, and I used that as a starting point for my pages. The wiki has saved me a ton of time, kept me organized, and serves as a great reference for me and for the inevitable next admin.
The only caveat is if the wiki (or the server it's on) goes down. This has happened once, and my instructions for fixing the wiki were... on the wiki, so extra troubleshooting for me. Thus, I find it good practice to maintain a hard copy of the wiki pages, especially the page that tells how to fix the wiki.
I'm running this on Redmine, which has proven to be bleedingly simple to use and administer, and much easier than trac, which we used before. It's especially nice having it on the intranet, as I'll just have a browser open to the wiki as I work on systems and refer to and update it as appropriate. It's very handy to document exactly how I performed a strange or experimental installation of some software that I'll want to replicate later without making myself crazy, and I'll take the extra few seconds to retype the commands I just used into the wiki from anywhere in the building, though I probably wouldn't do the same into a Word doc.
It's not so much the mundane day-to-day that I find that important to document. It's the weird fixes, the trouble spots, the command line parameters, the installation procedures, the changes that shouldn't have fixed it but did, and the horrific chain reaction situations that make one piece of software crash because a seemingly unrelated piece of software has the wrong version of the 64-bit library. Things that take 4 hours to figure out and 3 seconds to implement... those are the ones to document, and those are the ones that I'd be kicking myself 20 months later for neglecting to write down. In an afternoon, any schmuck could walk into the building and figure out which network cable goes where. Documenting the strange bits (and the frustrations), though, can get a malfunctioning mail server back up and running in 3 minutes instead of 3 hours (which, of course, is secondary to good administration keeping the server from going down in the first place).
-- I prefer the term "karma escort."
How exactly do you get a local admin account on a domain controller? I didn't think there was any such thing.
which is totally what she said
the cans of compressed air in every office supply store? inverted they throw out a very cold liquid that does exactly what you describe.
every day http://en.wikipedia.org/wiki/Special:Random
If the predecessor does write the passwords down, he deserves to be fired.
You can't always take it for granted. I've been on the cleanup-end more than once. Sometimes it's "engineered job security"/"they don't dare fire me", sometimes it's "I don't have time for that", sometimes it's just plain forgetting about a piece of hardware you set up the first week you started work there, and sometimes it's a legacy password issue. ("nobody's been able to login to that box since Phil left in '05") The rare treat is finding a mystery box that nobody knows what it does nor has any idea how to login to it. Too many managers don't understand the danger and consider it a waste of time or a bad risk to try to fix problems like that.
Sometimes it's an uphill battle when taking over, too. You want to document the settings on all the routers, but two of them are "legacy" password issues, and the router only supports hard reset (clears password, AND all settings) so you can't get the settings from it once you reset it, and you need the settings from it in case it gets reset. That's never fun, but you will find yourself in that catch-22 occasionally, and it's hard to blame someone for not fixing it because it's utterly unpleasant to deal with. (hint: get another router and program it how you think the mystery box is set up. swap. test. immediately swap the mystery back in. adjust the settings on the new one and test. Swap back out. repeat until you get it right, and don't reset the mystery immediately, keep it onhand for at least a month in case some uncommon thing requires a setting you haven't yet discovered)
Occasionally you can get lucky - contact the hardware vendor and see if the box has an undocumented soft reset. ("open it up and short together the two pads left of D-15, and you can login for 5 minutes with no password")
I work for the Department of Redundancy Department.
Reboot the DC into "Directory Services Restore Mode" (this is on Server 200 and above) and the local Security Accounts Manager is used again, not AD.
This is actually a way of resetting the admin password on a domain, if you ever need to. Boot from the Linux password reset CD (http://home.eunet.no/pnordahl/ntpasswd/bootdisk.html), blank the Administrator password (which resets the _local_ admin password), then reboot into DS Restore Mode. Log in and then copy cmd.exe over on top of logon.scr and reboot into "normal mode" (with AD functional).
Wait until the "screensaver" pops open and you have a command prompt that just opened that's running with SYSTEM privledges. From there you can run mmc.exe (or whatever else you like).
Oh and of course, this is exactly why you don't allow physical access to servers....
In reference to the PDC statement, I'm not aware of any way to use local accounts on NT 4 or below, but I think he meant just DC instead of "PDC."
Here's to the crazy ones