Slashdot Mirror
Writing Good Network Documentation?
_Hellfire_ asks: "As the Senior Technician at a small mobile computing service I have just recently wrapped up a rather large networking job. The network includes a Linux Primary Domain Controller, Cisco router, port redirection for VNC, tape backup solution etc. - all in all fairly complex and not the sort of thing that someone not already familiar could easily take over. I therefore want to write good, comprehensive documentation for other technicians in case I'm not specifically available. How have other Slashdotters tackled this issue? Where do you start? How do you make sure that everything is covered, particularly when you've spent the last three weeks looking at the same set of equipment?"
The format of the documentation is also important. Give docbook a try if you haven't already. Concentrate on the content.
No matter what you do, unless you spend the next 6 months writing a knowledge base on it, will you be able to document every little thing. Just try to make something easy enough for the typical *nix user to be able to re-build it if it gets totally fried and you were hit by a truck on the same day. Expect some calls for the time being until their admins get familiar more with the system.
/* oops I accidentally made a comment, sorry */
I overview the entire network, then overview the seperate systems, then go into intricate detail about everything i've done and why i've done it.
I do semi-detailed overviews to start because when I've needed documentation in the past, its been an emergency, and i've mostly needed it to know exactly what router X is connected to, basically how it is configured, and what it is used for, not the intricacies of the firmware pasted blatantly up front without any mention of what the router does.
do the documentation in HTML and hyperlink EVERYTHING that could possibly lead to a question or a footnote or any supplementary text. this is what hyperlinks are for but far too often i've seen network documentation handed to me as 5th generation photocopies that are near impossible to read. take your documentation to anyone that is covered by an NDA (if need be) and have them proofread your documentation. have him ask questions that he would ask if that network were handed over to him, and on his first day needed to find out why the WAN was down, but the wan router reported it as up.
then burn your documents on CD, and keep it close to your heart and any contractor that works for you.
that's what i do, but i'm paranoid. i've been the victim of poor or non-existant network documentation too many times to not be paranoid about it anymore.
Your done with any productivity. You plug it into a network and watch it crash constantly. You go to burn a disk and watch your machine become completly useless for the next half an hour. Your mouse dies and that machine becomes a very expensive space heater. Yeah you can blather on about OSX doesn't do that but when you have to continue to run an older version of Quark (god forbid anyone has to run this hunk of lard) to support your clients file formats OSX isn't an option. Did I mention how fun it is to troubleshoot network issues on a machine that comes with NO troubleshooting utilities by default. The *&^%ing things don't even come with link lights on the network card. Good luck determining if your IP stack is down or if your route is down or if you have any packets going out in the first place. Stick with you lamp analogy. The Macintosh was made for people that can not figure out much more than plugging in a lamp.
Just remember, no documentation == job security.
It also equals no advancement because you are too important in your current role. ie, your in a rut.
/* oops I accidentally made a comment, sorry */
first, label everything clearly. 2nd, document what you've done. print documents, in case there's no computer around to read them on. screenshots or logs of what you typed and have done is better than handwritten notebooks. 3rd, train someone in the Why and How of a system and it's response and they'll figure out the What on their own, usually. also, check out this article at kuro5hin and then do what seems more useful.
FreeBSD for the impatient.
In every tech's life, there comes a time when management starts to insist on better documentation.
Perhaps a round of layoffs or outsourcing is imminent. Perhaps the simmering disdain between techs and management has escalated into open hatred. Either way, you are clearly on the way out, and management wants to grease the wheels for your successor.
Objectives
You wish to produce documentation that:
General Principles
Concrete suggestions
More is not better
It is easy to produce impressive quantities of paper by documenting the very obvious and/or completely useless.
Spend lots of time on the appearance and presentation of your documentation. Your management is easily distracted by shiny things, and will not realize that your binders contain information that could easily be recreated by anyone.
Creating bad documentation is time consuming, and makes you look very busy. You can use your documentation as an excuse to avoid real work, or perhaps to con a few more days of employment.
One insidious consequence of overly detailed documentation is the maintenance required. Recording trivial changes is very time consuming. However, unmaintained documents quickly become inaccurate and misleading, arguably worse then no documentation at all.
Never tell "why"
Never describe what problems you've encountered, or how you solved them.
Do not explain what task each program or machine is doing. Do not explain the interactions of any systems, or which programs/machines depend on other programs or machines.
A programmer should never call attention to global variables, or functions with side-effects.
I've done more than a little bit of this in my time. I've learned that as long as it is well organized and understandable, you can never document too much. I prefer a layered approach that coincides with the different layers of the OSI model. I begin with drawings from layer 1 on up, basically from cable, ports, card/slot, box, etc. [Physical Layer] to DLCIs (Frame Relay), VPI/VCI combos (ATM) [Data Link Layer] to IP Addressing and routing information [Network and Transport Layers]. I create numerous drawings in Visio and write documentation in Word (yeah, I know, MS). When appropriate, I import my Visio drawings into the Word document(s). You can even structure you document using the layers of the OSI model as well. Hope this helps a bit.
Co-founder and designer at Music Nearby: http://musicnearby.com
If you want to see bad documentation, you should take a look at some of the fine products coming out of India.
We had a guy write a BizTalk adapter, which took him at least 4 months to make up one "portable" c file, which uses the Microsoft Windows 2000 Resource kit extensively, for programs like "sleep.exe". He wrote up 20 pages of documentation for this program, documentation that only makes sense to the only other Indian guy on our development team. He also made no effort whatsoever to make any kind of setup program.
For an example of bad documentation I point to his explanation of how you are supposed to use the Registry to set up settings for the inbound part of the adapter.
As a reward for his crappy work, my company is sponsoring his visa so he can come to the U.S. and work on our team. I can hardly wait.
Teach it to someone else and base the documentation on their notes, or even better have them write the documentation.
I'm not suggesting this as a lazy-man's solution or a way to to delegate all your work away. Your intimate knowledge of the installation makes some things seem obvious to you that would never occur to someone else. The person you're teaching it to is approaching it from the perspective of someone who knows nothing (beyond what's considered required knowledge for the position) and their notes will reflect that. You don't have to wrack your brain trying to figure out what the reader will need to know, your student will ask. If you really want it solid, have your student repeat the process with someone else.
The best documentation/procedures I've encountered were written this way, and it's become my technique of choice when faced with a similar task.
Under capitalism man exploits man. Under communism it's the other way around.
Especially in network documentation. Nothing beats back a snarl of wiring (even if you were neat, you know by the time someone else is going to have to work on it, it will be a mess) like some old-fashioned line drawings. And remember, only half the documentation is what you are writing down--the rest is in clear labeling of cables, equipment, and machines.
Aside from that, what some guy said above is perfect--a broad-sketch overview, a sort of narrative description of how everything works and fits together, and then detailed drill-downs in specific systems and sub-systems. No point in forcing someone to go through twenty paragraphs of detail when all they need to know is what Router A is supposed to connect to. The overview is often the most valuable part of the documentation.
I just use a basic outline format usually, but as long as there is a table of contents and some sort of logical progression to it, I don't think it probably matters too much.
Good documentation is what really distinguishes professionals from amateurs in this business.
No relation to Happy Monkey
I've just written a short book on this. Documentation for SysAdmins, SAGE/USENIX's 11th short-topic book, should be available shortly (probably at LISA 2003.
.@.
The network includes a Linux Primary Domain Controller, Cisco router, port redirection for VNC, tape backup solution etc. Thats complex? Almost any network would have Microsoft-based machines in it, therefore an Active Directory setup with some Linux servers in it, and a backup-tape system. The connection to the Internet is usually through a connection-sharing MS or Linux computer, or a linksys/netopia or cisco router. And I'm describing a small company. We have all that plus more, including ethernet, fddi and 802.11b, and a linux-based vpn and firewall using x509 certs. For changes, we just have some co-op or temp students we keep around with some experience who come in handy if someone major is replaced. We do have a large tree structure of documents as one big MS help file containing linked HTML pages and diagrams. It includes daily, weekly, monthly and annual procedures and license update requirements. The whole set is refreshed with new information continuously, and any joe IT guy gets a good understanding of the company's IT structure. The purpose of the document is not to explain technology, but act as a reference so it is based on bulleted points rather than paragraphs. It starts with the network technologies and OSes used, then goes on about the OS, patches, settings and major software installed on the 8-odd servers, and their purposes. An appendix lists all important contacts, licenses and security watchouts. This setting works with a company with one key IT person dealing with temp /co-op workers. If the organization involves IT managers and teams, then the documentation becomes horrendously complicated.
"Give orange me give eat orange me eat orange give me eat orange give me you." -Nim Chimpsky
1. Download all of the documentation locally, especially for all networking equipment. Keep all patches/firmware upgrades locally so that in the event of a failure you have everything you need onsite to solve the problem. I once had a router go down, and the only copy of the docs we knew of was online... Good thinking guys... :-)
2. Do an analysis of what would happen for each piece of equipment if it failed. A list of steps to detect the symptoms. Essentially, a list of things to help trouble shoot it. Do this, also for major configuration settings that could be wrong (Gateways, DNS, firewall rules, routing rules). So essentially list what you think it would act like if a switch broke. If the router broke. If the DNS server went down. What it would be like if the proxy server crashed. What would happen if the firewall settings got mis-configured. How they could tell the tape backups didn't work. Things like that.
3. Put in place a system that will help to pull the configuration off the machines. Creative use of SNMP, nmap and something like NetSaint (now Naigos) will help you pull the configuration off the running network to see if you can identify failures or changes in configuration. Document how the configuration should be, and check it using those tools with alerts sent out when something looks wrong.
Kirby
You should have started documenting before you started "actual work". This won't help you this time, but design the system on paper before you purchase stuff. That way, you'll have a plan set up that you can get other people up to speed with, as well as something that you simply edit when you decide that you spec'ed it wrong. Always start with documentation.
A true professional would already know all about documentation, but for those who don't, hereare some references .
but I read this on kuro5hin the other day.
I have found there are just two ways to go.
It all comes down to livin' fast or dyin' slow. -REK, Jr.
Those that write great documentation tend to get let go by their employer during cutbacks. "We don't really need Mr. Doe, he has documented his setup for us. Mr. Smith over from accounting could do it, for 1/2 the salary."
IMO, YMMV.
I'd like to second this comment strongly. Teaching the system to someone else (or better yet several someone elses) and using their notes is definitely a good way to generate documentation.
Using a wiki, I have found, is a great way to generate a collaborative set of docs. Like another poster suggested you want to use hyperlinks very liberally, something which is easy to do on a wiki. In addition, everyone can write stuff down and edit each others' writings to add clarifications, extensions, improvements, etc.
If you have enough people to make it worthwhile, you can also establish access levels on most wikis. The guru(s) and the tech leads can all write to the wiki, while first-line help desk folks get read-only access.
--Paul
Is this guys visa a h1-b or an l1?
... . If the team want to continue working there, get to know your upper management's motivation. On the other hand, if the team feels that the work environment just sucks, quit en masse (and before, just short the company's stock if your development team is a major part of the comapany).
If it is a l1 - they are probably going to outsource your entire teams effort to his Indian company.
If it is a H1-b, he will be a team member for a long time. Since he will be working for less than half the amount you work for, your job will on the line.
At this point, this guy has impressed your upper management with his technical abilities and/or price. Remember it is not your upper management who will be oursource -- it is you. Someone up there is saying, look, we can decrease our cost of development by 90% if we outsource our development to
There was this job I once had... it was a large project, and design work was mostly done and implementation had begun when I joined. I was astounded at the quality of the documentation: major design decisions and justifications, ideas that didn't work, things to look out for -- all documented. Not just that: it actually was updated as things changed! This inspired me to keep a diary of my first few weeks at work, which contained things like the organisation of the source code, the build system, environment variables necessary for developers -- the things that people already working on the project know but that might have been opaque to newcomers. Yes, we were proud of the system, and of the documentation.
Now we skip forward about a year, when it's time for annual reviews. One of the things that the moron manager dinged most people for was the quality of the documentation! I marched in and, in a high dudgeon, demanded to know the meaning of the outrage. The documentation was bad, he replied, because marketing and tech support couldn't figure out what the thing did. I tried to point out the difference between documentation for users and for programmers... I should have saved my breath.
There were other problems with this yahoo, of course, and most of the dev team quit soon after. The company went under a little after that.
nt
This complex network consists of one Linux PDC(you mean Samba), one Cisco router, one tape backup(possibly a second machine but, I doubt it)? You've got to be kidding, right?
Anyone that finds this network complex should not be allowed near the console! This is the most basic of internet connected network and identical setup can be found in hudreds of thousands of homes and small businesses.
None the less. Documentation is always a good idea. It should include a rundown of all services in use on the network including configuration files for these services, how the services are used and by whom and it should include a diagram of the physical network. If it actually starts to become complex in the future, using VLans and multiple frame-relay PVCs on a WAN then a logical diagram should also be included.
As soon as they have the blueprint, you become superfluous. They can ditch you for an ex-janitor cum MCSE at half your salary, or, better yet, for a curry slave in the subcontinent at one tenth your salary.
Not to sound too cynical [and not to imply that curry slaves and ex-janitors don't deserve a chance to earn a living themselves], but there's a reason employees do NOT document things. If the blueprint exists only in your head, you've got bargaining power. Once you've committed the blueprint to paper, however, your salary becomes a very juicy target for the bean counters.
... just make sure you do - I just started a job at a charity spread across multiple sites in the city, and 4 months in, I'm still finding things I didn't know about.
Big things as well, like an online centre half the size of the main one we run, which no one thought to tell me about until their router died.
You know, that explains a lot of the documention I've seen.
Coder's Stone: The programming language quick ref for iPad
I'm sad to see most of these comment suck. I hope this get through the signal to noise ratio.
Okay, here's the deal. There's not a whole lot of law out there to protect privacy. You could go after each Spamer, but that's like hearding cats.
That being said, they issued you a credit card. In order to do that they pulled your credit record. They were not authorized to do that. They are therefor in violation of the Fair Credit Reporting Act. Both you, and the credit reporting agentcy can collect money from the credit card company. Read the actual text of th FCRA for details and damage limits. Then send them a certified letter indicating you'll settle your claim for X dollars. If you can get a local lawyer to sign it and put it on their letter head even better. If anything you can be a real thorn in their side. Assuming you have a copy of the real application you've already won. You should write the credit reporting agentcies and indicate that company issued a card with out consent, and you want it removed from your credit record.
You may also consider reporting them to the FTC. Although with the current pro-business administration it may not go as far as it did pre 2000. If the FTC did decide to act it would cost them serious dollars as well.
I an not a lawyer, this was not legal advise.