Slashdot Mirror
Institutional Memory and Reverse Smuggling
An anonymous reader writes "Anyone who's worked in a large engineering firm is familiar with institutional amnesia. Things get built, and then forgotten. Documentation is supposed to help, but rots, is lost, or uses obsolete methods and notation nobody understands anymore. I recently found myself in a strange position, rehired as a consultant with the unofficial job of reminding the company how an old plant works. I even have some personal copies of documents they seem to have lost, which I have to awkwardly smuggle back in. I don't find these kinds of experience written about often, but I'm convinced they're more common than you'd think."
I have personally witnessed this happen with my former employer. Higher ups seemed to think that there was some benefit to the company "memory" or "team experience" if we almost brought a product to market. People would argue that at least we now had the experience to do this again in the future if we had to. Totally wrong. Teams were broken up and key players re-assigned. Some people left the company or moved to divisions physically remote from their former bosses. Any reports written could not be used to replicate anything useful and, to be honest, who would write a report explaining the history of an unsuccessful product? It never happens. Real knowledge is lost in a barrage of wasteful spending.
Sorry, but gray text on gray background is making my eyes bleed.
Yes, that's what makes it tricky. I wasn't supposed to have these documents, but some important internal drawings have been lost, and I have a copy. The safest solution would be to avoid bringing it up, I agree. Though I'm not blackmailing them for the documents or anything.
I've heard it happen several other times. I've had colleagues who consulted for a company, then a few years later get a call or email asking if they by any chance still have a copy of some old document. Of course they weren't supposed to have kept a copy, but if they accidentally did, it would be really nice if they could send a copy because the company seems to have lost it...
It mostly happens at lower levels of the org chart on a don't-ask/don't-tell basis I think. Lots of people keep personal archives, and sometimes the official archives have to get patched out of them.
> that's quite a different thing than the arrogant coders who feel that their colleagues are incompetent if they cannot follow uncommented code. if someone cannot follow my code then I feel I have done something wrong, not them.
IMO, _proper__ commentting discusses WHY you did something. If someone wants to understand HOW it was done, read the code, as you correctly reason. Any half-decent coder should be able to figure out what the code is doing.
EXCEPTION: If you are doing something tricky, then document that. i.e. http://www.codemaestro.com/reviews/9
Or just plain boobytrap the code all over the place with sanity checks and known gotcha conditions, spiked with good comments and explanations.
I ran across that just last week when developing an importer for iTunes music library - reusing an xml class I wrote 3 yrs ago, it blew out on me with a "you're being STUPID" sort of error message just under a dozen times during development. Opening up said code block revealed comments I made when I wrote the sanity check, describing WHY it had coughed up the message, and more importantly, what was causing the problem.
I of course had little to no memory of ever writing them, but there they were, providing me an instant catch-me-up in the why's of my code, and saved me from having to relearn all the internals and subtleties that went into writing that bug-free, beautiful, and time-saving class of code.
That's one of the great things about having a poor long-term memory - it forces you to comment (and liberally sanity check) your own code well, not because someone else will need it, but because you are going to need it. You get to be the direct beneficiary of your own good practices.
I work for the Department of Redundancy Department.
Management don't care, why should you? It really can't be all that much of a problem. More a perception than a problem. By all means go ahead and create/use a documentation system for your group but clearly there just isn't a requirement for anything more comprehensive.
All too true. The sooner you learn not to care, the sooner you will stop feeling the pain. Surrender to mediocrity; obey all rules and your managers without question. Above all, do not let your work interfere with your life. You will be a happier person, and probably get promoted. Listen to me, oh young ones...heed the words of the Ancient One: Despair!
Once upon a time, I worked in a relatively small organization within a Very Large Company. The VLC was renowned for its inventiveness, until a new manager came who felt that inventing things was much too expensive; it would be more profitable to just to put the word "Invent" on the corporate logo, and then re-sell stuff from countries who make things cheaply. Because our corporate logo was on these re-branded products, we must obviously have invented them. Thus, marketing ceased to be a partner of R&D, and instead replaced it. Clever, eh?
One of my contributions to the VLC was a simple document management system (DMS). My boss mentioned that all the engineers in our lab kept complaining that "they can't find anything"—specifically, nobody could find old documentation for code that was written by vanished civilizations of programmers that had previously labored at the lab. In fact, it turned out that this was a general problem among the labs of the VLC, and that they all wished mightily for a solution. Apart from really ancient projects (from, say 10 years ago), it was pretty difficult to find documentation for even recent projects, as each lab had a different way of organizing its documentation, different document formats, and no channels for distributing the documentation to other groups who might have a legitimate interest. So I brought in an easy-to-use and easily maintained DMS, pitched it to the engineers at our lab, and got them to use it. My boss then made me contact a bunch of other labs, and offer them the use of the same DMS. A lot of their engineers liked it, and there was much happiness. I got lots of pats on the back for what was really a pretty simple idea, and my boss loved me for it. A year later, I was laid off.
As I was the only one maintaining this system, and no one ever asked me to train a replacement, I doubt that this DMS—and its contents—survived, save perhaps as digital ghosts on some unlabeled backup media in a faraway storage cave. In a sense, I'm one of the perpetrators of information decay; I caused all the knowledge deposited in "my" DMS to get black-holed. Come to think of it, this is probably not a problem; I'm pretty sure they've laid off all the engineers by now.
I understood very well that simply setting up a DMS was not the cure for the I-can't-find-anything problem. It was a beginning, at best. What was needed was a comittment by the organization as a whole to make sure the information continued to be available for as long as the company exists. Because engineering practices change, so do documentation practices; documentation technologies themselves change—not to mention physical storage media and data encoding. My DMS probably should have been replaced with something better by now (if it had continued in use), and that means someone would have had to plan the transition, and move the old docs to the new system. Maybe the best thing to do is print everything as hard copy from time to time, and hire a librarian to keep track of this backup.
Considering how much has been said and written (and blathered) about "knowledge management", it's amazing how little attention is ever given to the temporal aspect of institutional knowledge. Every engineering company should have an Office of Knowledge that has as part of its responsibility not
Great men are almost always bad men--Lord Acton's Corollary