How Do You Document Technical Procedures?
ChadDa3mon writes "I work for a large MSSP type operation and we deal with a plethora of vendors, versions, and .... skill sets. We're facing a critical problem as we grow when trying to deal with these varying degrees of technical competency. The end result is we're getting to the point where we have to document every procedure and process, no matter how mundane or 'common sense' it may seem." How, ChadDa3mon wants to know, can complex skills be documented to account for various users? Read on for more details of what he's seeking.
"I've got a picture of how I'd like this to work in my head, but I can't find any software out there that seems to go along with it. I'm a big fan of keeping things simple, so I'd like to start with high level overviews. Each step in the process would be a general statement like 'Look for valid traffic on the monitoring interface'. For those who already know what 'valid traffic' means, it's easy to follow. However, if there was someone who was unsure what it meant, there would be a link they could click on that would pop open a new window (or something similar) explaining in detail what we're looking for and how to find it. It's my hope that using this process, people aren't just blindly running commands, but gaining an understanding into what that command is, and why we use it, what to be aware of, etc.
This seems like a job for a flow chart, but I don't like the setup of any of the ones I've used, such as Visio. It could also maybe fulfilled by a wiki, but there's so many out there I don't know where to start. I have to assume I'm not the only person who's facing a problem like this, so I'm hoping someone else out there can make some recommendations."
This seems like a job for a flow chart, but I don't like the setup of any of the ones I've used, such as Visio. It could also maybe fulfilled by a wiki, but there's so many out there I don't know where to start. I have to assume I'm not the only person who's facing a problem like this, so I'm hoping someone else out there can make some recommendations."
Documenting technical procedures is bad for job security.
I just use a text document with point-for-point descritions on how to follow the procedure. It's practical because you can print it and take it to the workfloor and cross the points you've finished. When you find out something new you can easily write it on the paper and add it in the computer file later on. Just make sure there is only one person maintaining the file, to avoid chaos and misunderstandings.
-- Cheers!
If there's a mistake or something in your docs are unclear, you want the guy using the docs to be able to fix it right on the spot. For that reason use a wiki, no-one is going to fire up Visio or whatever and diddle a flow char tin the middle of a call.
I assume the people using it are phone monkeys, make sure to track who makes improvements so that it doesn't penalize the guy who takes a minute to fix something but then their talk time suffers and they get bitched at.
I was reading through some of the Trac hacks for the wiki component and they had a folding plugin. If you create a table of steps you could then create a "fold" with greater detail should someone want to open it up and see it. The nice thing is you are not taken to a new page and you can continue to work and read the page you are already on. You can also imbed folds which can also allow a user to drill down to as much or as little detail as is needed or available.
All that is left now is to write enough information for the lowest common denominator.
---- Fight to protect your right to keep and arm bears! ummmm... ya I think that's right....
...I usually start with documenting the configuration on a Wiki (or a file in a Subversion repository somewhere) and then shift things over to Puppet when I get the chance. The nice thing about the latter is that you know all the setup specifications are correct since that's what's actually being applied to the servers.
Documentation of system provisioning is just one small part of the question you're asking, but hopefully that helps.
The Army reading list
That's what we call it. We write up a set of directions so simple even a monkey could follow. We include screen shots at just about every step so the user can see what it looks like. Currently we have them in a mass collection of Word documents spread over a bunch of network drives. Yea, it sucks to find a particular direction document.
We are in the process of loading all of our monkey directions into a wiki. That may work for you. You could create pages that have the high level overview directions for the users that know what is what and then have each direction link off to another wiki page that has more detail.
No matter what direction or option you find one thing you need to keep in mind is the more simplistic you make your directions the harder it is to keep them up to date.
Whatever you do, make sure the process for creating and updating the documents is simple, or it'll quickly become out of date and useless.
I would recommend NOT using any special software like a wiki for your primary documentation. It should be simple and printable, and not need a special server set up to access/use your docs in case of a disaster recovery situation.
Create a template document. If you use a word processor, set up standard formatting styles for sections, TOC, etc, so the docs are easy to create and navigate.
Put enough detail in the document so that you can hand it to a marketing droid and have them successfully complete the procedure. You'll be glad you did when you're stressed and under pressure from the Big Boss in an emergency (or if the whole IT department keels over from a tainted batch of Mt. Dew, and the only people left are completely unfamiliar with your environment).
That's always the catch, isn't it? I'm sure we've all seen the printed instructions with more handwritten addendums and notes in the margins then there is printed text.
Proud member of the American Non Sequitur Society. We might not make much sense, but boy do we love pizza!
but I can be bought.
You need a competent process engineer, and good ones are expensive. Bad ones are a dime a dozen, and will drive your costs up by insisting, for example, that every procedure be a documented procedure, that every process needs to be flowcharted, that there's a distinction between a process and the document describing the process, or that the fiction that is RACI is not reducible to a single directive: "accountable".
This is, as they say, job security for the process engineers because they're constantly chasing a moving target. Also, the instant employees realize or suspect that they're being made interchangeable by participating in any process engineering effort, they'll learn to conceal key details which will make all work to date useless.
Organizations that don't realize that have chosen the way of pain. Yours is probably one of them.
Slashdot is my Mercer Box.
Speaking as someone who has a degree in technical writing, your question is symptomatic of the entire industry's attitude toward (read: lack of respect for) technical writing. You hire people with degrees to run your networks and program your computers and execute your business processes, so why are you not hiring someone with a degree to do your technical writing? It's a specific skill that most people do not have and those who do it have had to study and learn. Realize that and you'll be better off. [/rant]
"We can categorically state we have not released man-eating badgers into the area." - UK military spokesman, July 2007
I can't stress this enough. There are professionals who have post-graduate education and vast experience documenting and communicating technical procedures. Hire one of these people.
If you don't hire a technical writer you will face all kinds of problems. You'll have technical people with poor English skills writing incomplete directions because they make assumptions about what the reader knows. You'll have 50 manuals with 50 different writing styles. You'll have 10 instructions in one sentence with no commas. You'll have unfortunate typos and grammatical errors which change the meaning of the sentence.
Technical writers are both technical and writers. Hire one (and not the cheapest one you can find) to do the job right and chock the expense up to preventive maintenance. The alternatives are putting faith in poor documentation and, in the best case, spending needless cycles working out the kinks, or, in the worst case, spending needless money on a settlement to your injured customers because your staff were improperly trained.
Choose any goddamn thing to start documenting (I use MediaWiki, since everybody seems to have some experience with Wikipedia nowadays, so it's not so jarring). Job swapping is essential, since you'll never know how good your doco is until you test it. Choose the best communicator with skillset A, the best communicator with skillset B, and let A do B's job with B watching over, documenting all the way. Swap and repeat. Do the same thing with all other combinations of skillsets you've got. Then test again: when A takes a day off, find a B to replace him/her as a stand-in. See how well he does. If it's not tested, it's useless.
For simple procedures, I just use Powerpoint, and have extra, separate graphs when a particular task can be expanded to a subgraph.
If anybody has any other graph packages to recommend, I would really appreciate upgrading.
The disadvantage is that you've generated read-only documents.
Comment removed based on user account deletion
If you are mainly a Microsoft shop, SharePoint works will for posting procedures and related docs. Works pretty good for us. Is a pain to use when trying to rearrange items later, though....
You're messin' with my Zen Thing, man.....
I started doing a little software development for my company (when I was hired to do help desk), and I was once asked to write a procedure for that. How do you write code. I replied -- Step 1) Go to college for 4 years and get a computer science degree.
It really doesn't matter what you decide to use. Wiki, Sharepoint, a file share with a bunch of Word docs. It doesn't matter. Just get things written down.
Now, that being said, I tend to mix procedure docs with configuration docs. You can try to keep those seperate but it's often easier to combine the two. So, say you have a thin client system set up, you can have one doc that documents how the system is configured and how to perform basic tasks.
You don't need to get too detailed, of course. The intended audience isn't a non-technical user. Each document should have a few basic sections; Revision history, purpose, intended audience, and a simple explaination of terms such as for certian acronyms used. It's also useful to include software versions so you know what version of the software the document was written for. As you upgrade the softwasre, update the doc and update the software version.
Create a document template and with pre-formatted styles. That way, you cab bust out organized documents that all look the same and every one will automatically get things like a TOC based on your styles. It's worth putting in effort here.
In the end, though, just get as much information as possible down on "paper" and then work on making it accessable. I'd rather have to hunt for that IP address or login to a web site than to not have it at all.
And remember, it's not your job to provide TRAINING materials to people in the form of this documentation. (unless it IS your job, but it doesn't sound like it.) Your job is to make sure that the systems stay running and if something should happen to you, the company isn't screwed because the systems are documented. Perhaps more importantly, if a server you worked on a year ago crashes, you'll have all the information you need to fix it.
Anyone that thinks Documentation might lead to their dismissal because "We don't need him anymore" is dead wrong. If you write documentation, you'll be the most loved person in all of IT in your company.
- It's not the Macs I hate. It's Digg users. -
Postits Everywhere.
What's in a sig?
Must be some kind of new-fangled term out there....haven't heard it or seen it really much out in the work place...
Seriously, I actually have RARELY seen it in the tech end of things. On projects I'm on, you have those people doing risks, and other paper pusher jobs doing all kinds of documentation, but, I rarely see much of any kind of good documentation in the tech end of things. I'm talking big projects, govt. projects...etc.
For instance now, working on a gig to take over a number of database instances....I didn't even really get a list of what databases instances are even OUT in the environment, much less how each was set up, etc. Hell, took me weeks to find someone who had the oracle OS user password.
I'm currently documenting stuff, but, I don't find it that unusual to come into a job, and in the tech area actually have very little documents and procedures. I always hear about people out there going through and heavily documenting things and processes, like naming standards....but, very rarely have seen it in real life practice.
Light travels faster than sound. This is why some people appear bright until you hear them speak.........
Actually, if your workplace is ISO certified, it would have had to document procedures and work instructions to get and maintain the certification. A procedure contains high level information, while a work instruction actually documents what steps it takes to do the job. The one thing ISO does is force (theoretically) a company to do what they say they are supposed to be doing, the other thing is does is generate tons of documentation. It just goes with the territory if you want the cert.
I worked very briefly at Dell, doing corporate tech support (hardware), and well not to brag but I was still #1 in the stats ranking a month after I quit until my averages rolled off... anyway I spent much of my off-call time trying to figure out a way to condense my smarts and experience into easy-to-follow instructions for my neighbours, who were often struggling with what I considered very basic problems.
To put things into perspective, my average call time was 5 minutes. That includes the occasional hour-long clusterfuck, which means the great majority of my calls were under 5 minutes. The top 10 techs had averages in the 15-20 minute range, and most everyone else was 45 minutes and up. Well now what was I doing differently, besides being the guy with the most hands-on PC experience ? I was being lazy, that's what! And I was committed to sharing my lazy ways with the rest of the crew.
During the exercise. I learned a few things:
1. It is damned hard to put into words things that are trivial (to you)
2. Logic, much like common sense, is a rarity these days
3. Most people fail to factor in the true cost of support time
So for #1, I had a non-guru friend take instruction, helping me dumb things down and bridge the gaps until we both agreed he had mastered the situation. He would express his frustration at my poor instructions, and ask a zillion questions until I had a grasp of my own internal thought process, of things that I did automagically.
For #2, I ran through a large number of scenarios and wrote down my own inner thoughts at each decision point. At the end, I trimmed these down to a rather large, multiple-entry flowchart. The neat thing is it covered both specific problems like "my hard drive is dead" and fuzzy symptoms like "my screen is blank". You would start at the edge, identifying the main symptoms and then you followed the paths until they crossed. At the very middle was the final solution "Replace entire system", the catch-all in case no other endpoint was satisfactory.
For #3, I dug up details on the approximate cost of support time. This included salary, utilities, and amortized equipment cost. Then I made a list of common support resolutions and their total cost (parts, labour, shipping). This allowed me to show how a long call can actually be more costly than replacing the computer in its entirety, especially if the extensive troubleshooting still leads to component replacement.
So in the end, we had verbose instructions, a troubleshooting cheat-sheet, and a cost guide. It may have enabled incompetent techs to perform tolerably, but more importantly it gave everyone the tools to learn very quickly. After a week or two with these troubleshooting aids, many people had the common issues memorized and had developed strategies of their own.
That's what I consider a successful transfer of knowledge. Knowledge is much more than just static information, it's also the process to create more information.
-Billco, Fnarg.com
I used to be in the troubleshoot everything camp, now I am firmly in the copy the user's crap and re-image camp.
I also found this PC boot failure flowchart to be of immense help.
http://www.fonerbooks.com/poster.pdf
Beyond that my procedures stuff is pretty high level, stating what is required and letting the operations folks implement it how they like.
The nitty gritty stuff I have had to write has been pretty specific i.e. DISA compliant CentOS box running Snort, and that's because the DISA and NSA docs for Linux aren't current to RHEL 5.2 and the guy who's Snort install doc we were using died.
http://www.internetsecurityguru.com/
However I am thinking a Wiki would be sufficient for most folks needs, I like Mediawiki, it was one of three packages that came with my webhost, and it has been easy to use.
I work on the teach a man to fish theory most of the time, and ask my analysts and other coworkers to come to me after Google not before. But as a rule of Thumb if you had to find the answer to a non trivial problem through Google it might be a good idea to copy and paste it into the internal wiki rather than rely on you remembering your search terms, you being at work, and your internet connection being up the next time it breaks.
Not the way you're thinking. Where I work we use a wiki (mediawiki) and it works as well as any system I've seen, probably better than most. Now for the "can't be done" part:
Tasks like building a server, installing some software, connecting cables, etc., can and should be documented. Troubleshooting however cannot. Troubleshooting takes profound understanding of all the parts surrounding an issue and how they inter-operate; the ability to think long, hard and meaningfully; the ability to devise experiments, execute them, collect and analyze data and arrive at meaningful conclusions based on all that -- the scientific method, in other words. This is the kind of job that humans do better than machines and it is proverbially difficult to even understand, let alone document -- witness the "expert systems" of 20 years ago.
The big problem is that all non-technically-inclined people, including most people in management positions, think what we do is something mechanical akin to what plumbers do -- no offense to plumbers, who do things I can't do. From a user's point of view, "I can't print my e-mail message" is not much different from "I can't wash my dishes". Locating a clogged pipe and clearing it does require some technical skills and can certainly be documented. The e-mail printing problem however is several orders of magnitude more complex than that and, although it's probably possible to write a flowchart to document any possible thing that could be wrong in this scenario, it would be ludicrously huge, silly really. And that's talking only about day-to-day. If I'm really good at what I do I will design and implement systems so that errors of that sort are minimized. In other words, we're being asked to do the job of civil or hydraulic engineers on a plumber's salary.
There, of course, is the crux of the matter. Microsoft in particular, but the whole industry actually, are guilty of selling to managers the idea that this job can be done by drones, at all levels. Management is happy because drones don't cost much. Then, of course, drones are all they can get for what they offer.
What you want is knowledge, knowledge that can be provided by some, but by no means all, not even most, of the people who work for you. And you're tired of seeing those few good ones go away, leaving mostly drones behind. The only advice I can give you is: do document what can be documented. Make it a priority and make everybody responsible and accountable for documentation. But do acknowledge the really knowledgeable people in your team, treat them and pay them just as you would any other kind of expert. I know you're probably not in a position of paying your tech guru at orthodontist levels, but try to treat her at least as well as you do accountants.
You saved me some typing!
The older they were, the better they were built.
This is because of the building code.
Today, builders build to code, and the code specifies bare minimum safe method. As new techniques are developed to cut costs, the code is modified to allow cheaper, less redundant methods.
Before the code, builders built in competition with each other. They tried to build better houses, so they would have better reputation, so they could charge more.
But now, they just build to code. Full stop.
There's a place near me in PA that has no building code. All the buildings there are fundamentally stronger and better, no lie.
I try to apply this to programming. If a programmer doesn't write code good enough to stand without external documentation, I find another programmer.