Lux Interior asks:
"Help! I am the ad-hoc computer guy in a small satellite office of a larger company. We have no CIO, no IT department, and no policies whatsoever as regards data retention or backup. Therefore, a lot of company property exists one place-- on individual hard drives. The office is made of almost entirely of rudimentary users, on WIN98 and 2000 machines, who never, ever, back up any company information. Has anyone out there had experiences in a small-office setting with: changing users' behavior in regards to managing their data; setting up best practices for backing up information properly; and making sure that the most computer-apathetic users comply with what you've put in place?" Sometimes the best way to make users conform to policy is to not give them a choice in the first place. Automated backup systems on each workstation can go a long way in helping this. Which software packages have such functionality (the more unobtrusive, the better)?
"Several weeks ago we lost six years' worth of extremely important data on current and continuing projects that not even a data recovery service could get back. As a consequence, it is now my job to make sure this doesn't happen again. I have an offsite data storage service retained, but now, how do I get people to back up their files to our file server so I can back up our data from one location? (Also, having the data backed up on our file server of course means that most inadvertent deletions can quickly be fixed in-house).
This is all taking place in a Windows environment, with an NT 4.0 file server, and I am far from an experienced Sysadmin.
Fun, Fun, Fun.
Any input from slashdot readers would be great, and save me much dyspepsia, insomnia, and general hassle."
Slashdot Mirror
I wouldn't call this "easy" but there's plenty of sites that should have instructions on re-usering people to log on to the fileserver as a domain server and resetting their my documents folders to be on the file server... then it's just going around and mopping up. Then you back up the file server and tell people if it's not in 'my documents' then it's gone gone gone and its their fault.
Automating backups on workstations, very bad, very difficult. For example, getting people to not turn them off, or even turn off a power bar (wake on lan doesn't work so good in such a situation.) What you do, if you're on NT, is set your system policies so that my documents, all that stuff, is on the server, in their home share. Tell them to put everything on home share. Tell them that anything that's on their hard drive, and lost, will be their responsibility. Explain why they need a central data repository. MAKE DAMN SURE YOUR BACKUPS WORK, OR YOU'LL LOOK LIKE AN IDIOT. Then, if need be, pick a sacrifical lamb. Back up their harddrive, then engineer a 'hard disk failure.' Make sure something important was there, that wasn't properly placed onto the fileserver. This'll drive the point home.
Vintage computer games and RPG books available. Email me if you're interested.
If the user doesn't want to be bothered backing up their data, make them sign a waiver absolving you of responsibility when (not IF) disaster strikes and s/he loses vital data. At least then, when they're angry and upset and looking for a chump to take the fall for their stupidity, you've got a convenient ass-cover with their autograph on it, and it won't cost you your job. :-)
~Philly
Build yourself a samba server on your favourite brand of UNIX (I prefer Freebsd, many don't). Attach a tape drive and use Amanda as your backup program. Get them all accounts on this machine and get them used to using their "Z:" drive for everything. Then everyone has a daily backup and you are in control - which helps a lot when dealing with people less technically competent than you.
Policy doesn't work until something really bad happens and someone with real power in the company says "Do it or you're fired".
I've been in this situation with dozens of companies, and policy only takes root when error rears its ugly head.
Sometimes the errors cost headaches, sometimes they cost you a lawsuit.
My $0.02 will always be worth more than your â0.02, so
to the server. That will handle a large chunk of the problem. The rest of it is just go into the programs that you use and map the preferences of the default folder to one on the server. If that is not an option, then you have to teach the users what to do and if they don't follow the policy and data is lost, it's thier problem.
There's a utility in Win2K, (intuitively called the 'backup' utility), that allows a setup of recurring backups (incremental or full). If you can get access to your users' computers, you can run the utility to backup specific folders on a regular basis, keeping them completely unaware of the process.
:)
It's hard to find, so if you go to 'help' under the start menu and do a search/index-search on 'backup' you should find a link in the help pages to the 'backup utility'. I've found it to be quite handy.
As for win98, well, Good luck
If enough users are Windows 2000, format the drives as NTFS and lock them down. Don't allow them to write files to their own directories. Obviously, some concessions will have to be made for application use, but they probably won't find their way to the few directories where they could theoritically save files. Give all users mapped network drives, both personal and shared, such as H for their home drive and S as a departmental drive, K as a common drive, etc. Again with 2000, point their My Documents folder to their H drive, making them save stuff to the network without realizing it.
Now, get a good back-up scheme on your file server, which I assume you already have, and you won't have anymore data loss problems. It also removes accountability from you. You manage the computer systems. Tell users that IT is simply not responsible for data lost off their local HDs. If they ignore you and then lose data, shrug your shoulders and point to IS policy. That'll learn 'em and learn 'em fast.
In a situation like that, you're probably going to find that the users are so accustomed to keeping data on their hard drives that they simply won't back anything up on their own. Most will simply refuse to do it.
Even more unfortunately, an automated backup system probably won't help either. With a mixed environment like that, you're going to find that most users have data scattered all over their hard drives, making it virtually impossible to backup anything less than the entire drive. Not a good idea if you're dealing with any more than 5 to 10 users.
The best idea in this case would probably be along the lines of having each individual user move all their important files into one directory tree. Don't even tell them it's for the purpose of backing up, because then they probably won't do it. Make up some story about viruses destroying random data, and tell them that this is part of a prevention method. It sounds totally idiotic, but they'll believe it...instilling fear is usually the best way to go. Once they've moved everything, make sure they know to keep all their data files in that same directory tree from then on, and set up a scheduled backup on each machine - hopefully to a networked tape library or network drive of some sort.
Give this a few months, and get them used to keeping their data in a very specific place, and then start giving them network storage space to use instead. It will make the backup process a lot easier, and you'll be able to do it daily instead of once every week or so.
In any case, you have to approach this very SLOWLY, or you'll freak them out and they won't be any help at all. Baby steps, man...baby steps.
So, you have two options: (1) If you have reasonablu fast network connections, take the choice away and install automated workstation-to-server software that runs every night. This won't work for roaming laptop users though. (2) Hold a series of "computer training classes". Say 4 or 5 half-hour classes where you teach e-mail ettiqute, tips and tricks, Internet searching, that sort of thing. Make them mandatory (you can usually finagle this through the HR or Training group of the parent org). At one of the classes, discuss backup, then pull out a form stating "I have attended the backup class and understand the consequences of failing to back up my work. My department and I accept full responsibility for failing to use backup tools provided". Require them to sign and turn it in (again, HR and Training will usually help with this). Send copies to the department heads.
That won't prevent data loss (or even the loss of your job if something goes wrong!), but it will help somewhat and also get at least some people thinking.
sPh
We used to have the same thing in my office, and I was in the same position. We setup a network file server with lots of hard drive space, and forced users to logon to the domain. Then we secretly replaced their hard drive with 4 gig versons, so there wasn't much room for them to be saving on. And then we setup a nightly backup of that entire server. Withing 2 weeks, pretty much everyone had their stuff on the network, and as an added side effect, we came up with a naming convention for saving client data, and all client data is saved in the same place now, making finding info much easier and much more efficient. Every user also has a don't ask, don't tell, personal folder that only they (except me, of course) have access to. For personal stuff, like docs, pictures of family, mp3's, pr0n, you name it, they got it...
Hope this helps
Th
You buy a TAPE DRIVE. Do not buy a cd-rw. Buy TAPE. Get DDS-3 dat, or VXA, or AIT, or DLT. Make your life easy, buy an autoloader. Make sure retrospect supports it (they have a list). Buy enough tape capacity to back up all the files on all the hard drives without you having to sit and change tapes for hours on end. Retrospect will automate the entire deal so you just need to be there to pick up the pay check.
Someone will give you grief about the cost of tape drive. Tell them to FUCK OFF. Tape drives are CHEAP compared to how much it'll cost your company to LOSE DATA. Buy LOTS of tapes.
With this, you back up EVERYTHING. The first part of a new backup is a pain, but from then on in Retrospect will just back up the changed files, making life very easy. Use multiple tape sets and rotate so you always keep a couple of good backups around.
I can't stress this enough, back up EVERYTHING. Do not say "I will only back up 'my documents'". People save their files all over the damned place and never know where they are. They delete stupid system files they didn't know what they were for. Once you're over the pain of the first full backup, which can take a couple of days depending on the size of your place it's easy and the incrementals are fast. And you can do DISASTER RECOVERY. As in "my hard drive crashed and I lost everything, please restore my computer to the way it was". You can point, click, blast everything onto a new drive in the machine and the machine runs exactly as it used to.
Do not wait for disaster to happen to try this, after you get a backup under your belt. Go through the restore procedure. Get another hard drive and practise doing complete system restores so that you can do it in your sleep when your CEO calls you at 2am to find his deleted girlie pictures.
If you can do this, your cow-orkers will love you. Women will love you. Men will want to BE you.
Dramatic maybe, but I'm a damned happy user of Retrospect for years and it has saved my ass more times than I can count. There are other products (Backup Exec) but I have not used them, and so I cannot vouch for them. I use Retrospect every day.
Sadly, in the environment I'm in this would get the IT guy who did it canned.
Before you try anything like this, make certain you can outmaneuver anybody's political BS. One phone call to _your_ boss by the right person could be very bad.
Politics around here is so bad that one department bought a single (external parallel port) ZIP 100 and managed to get the IT folks to send someone around once a week, moving the ZIP, to back up every machine.
Your best bet, get your superior on board no matter what you do, and get your a$$ covered on paper.
You HAVE to enforce this; put a policy in place that all critical data must be stored on the central server; any locally stored data is not your responsibility and cannot be recovered.
With that done, set up your backups with relevant retention/rotation and go from there.
The only possible spanner is if you have a slow network and the users need large files and they complain about performance. In that case, use the "My Documents" folder and centrally store the network profiles; that way they'll get written to the server on logout and can be backed up as normal.
First, write a policy for users to follow (not more than a page), get your boss to sign it, and then distribute copies. This tells everyone that the boss is behind this. If your boss does not have line authority over the people in question then get someone who does.
Second, get your boss's approval for a half hour tutorial for all staff on the subject, attendance mandatory for all users including him/herself. Get the boss to start with a brief repeat of the disaster story, then hand over to you (write the boss a script if necessary). Tell people why they need to do it and what it is they need to do, but obviously don't go into techie detail. Also emphasise that unsafe behaviour is letting the team down: its not just your work at risk, its everyones.
Its your responsibility to determine policy, configure machines, tell people what to do, monitor progress, and report to your boss. This can and should include saying that certain users are refusing to following departmental policy. Its then his/her job to take things further, upt to and including disciplinary action if necessary. Its not likely to be necessary: few people are that boneheaded.
Good luck. Culture change is hard, but its one of the most valuable things you can do.
Paul.
You are lost in a twisty maze of little standards, all different.
This story reminds me of my earlier years as a indy IT contractor...
One of my first customers was a NAACP trial lawer. Every day one of his jeloppy win95 machines would take a shit, and every day I would fix it. I would constantly remind him how he needed NT workstation instead of 95.
When it came time for billing we would always go around in the same circle, "Now Wobbert (he had the elmer fudd ebonic accent) Why am I gettin chwarged 5 times for the same fix?"
To which I would respond, "Because you broke it 5 times!"
Then his killer statement would come, "Well then shouldn't you have fixed it right the first time so it wouldn't break again?"
To which I would go into why 95 was a POS and NT4.0 was alot better. Then we would go into costs and I would show him how his long term maintenence cost would drop if he made the switch. It never really registered with him though.
I also made the pitch to him about having a centralized server for his employee's to store data on. He just could not understand that this computer wasn't for the employee's to use nilly willy as they pleased.
I feel for you man, just run while you still have some sanity left.
You plan is good start, but is incomplete. Here are the bare minimum (IMHO) requirements to make your plan feasable:
1. Rotate tapes on/off site weekly. This means two complete backup tape sets. There are many services that will do this for you, check your yellow pages.
2. At least quarterly, one complete backup (not incremental or differential) tape set should be pulled from the lineup, and placed in long term, off-site storage.
3. If you don't want the hassles of #2, use a real-time backup utilty like NSIsoftware's DoubleTake (I used to work for them), there are also other similar products. This method requires a second computer with enough disk space to hold all your data. It's a different hassle, but perhaps easier to deal with.
4. VERIFY your backup system at least monthly. Read back data from the backup media and perform test restores.
5. DOCUMENT your plan and proceedures. Keep the documentation in several places, on-site and off so that in the event of disaster (server, site, or responsible person) others can retrieve the information.
If you take posession and responsibility for your users' data, YOU will be blamed if it is not recoverable when needed.
Article X: The powers not delegated... by the Constitution...are reserved...to the people
After many years of painfull experience, I have only one suggestion:
**** NEVER EVER UNDER ANY CIRCUMSTANCES BACKUP WORKSTATIONS ****
Got my point?
Instead, go out and buy a cheap server. You can get a AtlonXP 1800 w/ 512MB of RAM and 100 GB of disk for around 200-400 dollors if you put it together yourself. From there map all drives. If you ever get stuck in a situation where you back up the PC's
A) It will get difficult to wean users off of later.
B) Builds bad saving practice. To comply with document policies, you really must centralize where your documents are.
C) Backup software tends to fail/hide/be to verbose when too many boxes are used.
If you absolutly have to backup workstations, look at network backup products like Veritas or Seagate software (err. they may have sent the product to some other company). Ask a user explicitly for a single directory to backup.
For servers, a image level backup is always a good idea. It tends to be the different between 3-4 hours recovery time and 12-24 hours.
Anyways, that's my advice.
It is actually the way a lot of large (and I mean 100,000 desktops large) companies essentially do it. Create one 'Build' and roll that out to users. All data (Lotus Notes data, word docs, etc) is stored on servers.
Then when Level 1 support can't fix the problem by doing a 3 finger salute (Ctrl-Alt-Del) then they simple redeploy the build.
/b
[Please type your sig here.]
Dude, attitudes like that are the reason most people, given a dark alley and a baseball bat, would beat the living fuck out of their average IT person.
/with/ people. Will make your life a lot less stressful.
How many people actually enjoy working in an environment where coworkers in other departments are comeplete assholes?
What if your HR people, when asked about specific personnel issues (pick any one embarrassing or annoying medical problem), basically answered with a raised middle finger?
Work
Why not try making the staff's compliance public? Test the staff's own backups to the server periodically / randomly. Then fire off a memo to the boss stating the results of the test; who complied, who failed to backup as of the night before. Post a copy of the memo on the frig. The staff knows they've been ID'd, and in the event of a true failure; the results can be trotted out who does and does not try....
they let me centralize our files. An office full of CAD drawings and 3D models can get screwed up pretty quick when a drive goes down.
We put in a 240 GB NAS box, stuck all of our data on it (it only filled 20 gig), and I run automated backup on it every day. Had to invest about four grand in it, but it was worth it. Case in point - our accountant (boss's wife)'s drive died yesterday. Gee, whats all our billing information worth? Think maybe we paid for that system in one day?
anything i tell you will cloud your opinion.
A twist of a thumb screw, a poping off of the side pannel, an unplug, and putting it back together is all it takes to convince most users that their hard drive has crashed. Then take their machine away from them, give them a blank machine for a couple of days of sweating it, then bring it back talking of all the heroic measures you've had to go to.
Now you're a hero for saving their data and you've driven the same message home.
If you want to do it to the entire office in one go, come in one night, do it, then tell them that there was a power surge. Fix the machines belonging to people who control your promotions either impressively quickly, or sufficiently slowly to convince them that what you do really matters to them.
Then, in the middle of the night, format everyone's machines and stick fresh OS installs on all of them. If possible, ghost one machine's fresh install and use it everywhere. Then, the only backup you have to worry about is the H: drive.
You're offloading system administration tasks on the users, and giving them an drop-dead ultimatim. Not cool. No fallback. You'll cause much harm.
Instead try billing it as an "upgrade". That way they'll take any inconvenience as a side-effect of something useful to them, rather than as you deliberately screwing up their data and lives to make your job easier.
Also:
Do it by departments, workgroups, or segments of the cube farm, in stages.
Start with a very small group. You get to work the kinks out with a minimum of trouble if something went wrong, and the group will spread the word to other users on how to ease the transistion. That will let you do larger groups later.
Don't just format their disks. Swap 'em out for fresh ones and keep the old disks handy. Help the users recover any data from the swapped out disks for a few days, check that they've got all they need, maybe back the disks up just in case. THEN format them and swap them IN on the next group of victims.
Make a point of how much extra work you're doing to be SURE they don't lose any important data during the transition (even though you're not doing all THAT much extra). And of course harp on how the main point of the upgrade is to protect their data in the future (which IS true).
Bantam Dominique roosters crow a four-note song. Once you've heard it as "Happy BIRTHday" you can't NOT hear it that way
Sadely, the first person who will fail to back up their information will be a VP or a sibling of the boss, who can't be fired.
You cannot be "given" the ability to ensure that people back things up, all you can do is ensure that your file servers are backed up & restorable. Users must be responsible for their own data. If they don't back it up, and it gets destroyed, see if they can fire the idiot.
Karma: Food Fight (Mostly affected by Date Plate).
For small and home offices without a server and tape drive, I have had good luck with a CDRW drive and Stomp Backup MyPC software. A predefined backup job can be scheduled or manually started by the user. While CDRW are not large enough in capacity to back up an entire system, it's good for backing up My Documents, email and Quickbooks data. Another option would be to have them work in a terminal server session running on a server at HQ over a WAN link. This requires a more complicated printer setup and is no good for running mission-critical apps because we all know WAN lines are not always up.
OK, yes, you've covered all the sysadmin geeky things you can do for file backup.
But there's something much more important in the long run. (Granted my following point is about document retention, not just 'backups.')
Keep REALLY GOOD PAPER RECORDS! Seriously, most of your users' most valuable information is in some simple document form. Documents they wrote up for customers. Financial documents. Have an appropriate person (lackey, with operations manager supervision) collect the most valuable documents, make sure they're printed and filed in triple, and send them off to Iron Mountain and two other safe places.
We'd love to think that digital media is the most portable, flexible, yadda yadda. Well, it is...sorta. But it's also quite fragile. Sadly, there probably WILL be Word 97 translators out there in the year 2020...but they'll probably be sorta crappy. Paper records are really quite valuable.
Oh, and you think 2020 is far off? Ever do legal research? Read up on deeds and stuff? Documents need to last a long time.
I had a similar problem at the school I work for. We have a central file server running Vertias backup exec. Gave everyone a "home" directory on the server and mapped the drive via login script. I told everyone to put all their documents there....and of course some people didn't do that.
So I forced the issue by installing Fortres on every machine and locked down the user's ability to save files locally....so if they want to save the document, they need to save it on the server. I am also running the central control module to easily push out changes to the workstations.
I thought about using roaming profiles (Windows 2000/NT/9x required), but they are a pain in the ass. Some local apps don't like the roaming profile thing, and the file synchronization on login and logoff was killing our network bandwidth (as well as annoying the users with really long login/logout times.)
Hope this helps.
-ted
One of the big problems with Windows is that there's no standard home directory. Microsoft apps save everything to My_Documents, other apps to wherever they feel like. Even worse, sometimes it's really hard to find where an app has saved its files becuase the directories are cryptically named. Then you have the problem of people moving their data to places that make more sense, and other people moving them to somewhere else entirely.
More than once I've had to rescue a small business who moved their Quickbooks data into My_Documents, then their accountant worked on it and saved it back to the original location. Anyone who's worked with Quickbooks knows what a mess this is- you can't just merge the two files. It's back to square one- sometimes with weeks' worth of data!
If Microsoft and Windows developers would just standardize one one home directory, it would make everyone's lives a lot easier.
Or insert a little time bombs into their computers, and when the office is on fire, appear with the water, save the day, become a hero, tell them that as a hero you know what's good for them — daily backups that is — and get a rise for saving not only the hard drives content, but also your coworkers.
But seriously, I don't have much time to read every +5 Insightful conspiration plan as well as the real solutions, so I'm risking being a little redundant.
You might of course try making them do daily backups, but they won't do it for sure, even if it means that every employee has to use 20 minutes every day. And they're right, like they're not changing the oil in the company's cars. They want to have computers which let them do their job.
The simplest solution would be to use Samba servers for users' files storage (I don't know if NFS work under Windows) &mdash which will act as a remote storage of everything your coworkers do, in a way totally transparent for them (just another directory on their computers to which they should save the important stuff) see Samba.org for details. If it's a small office, you just need a single file server for that so the hardware won't cost you much, the cost of software is $0 (or you may use Microsoft sollutions if you have lots of money for that — ask someone who uses NT file servers for more info about the MS way).
Now you have every important data on one machine. You can set up this machine to automatically sync the main directory with the redundant copy of everything in a second (or more) directory, so when someone deletes something important, it's still in the second copy, or third, etc.
But now you have a single critical point where everything important is located — that's to risky. You should have another machine, in another place, which will sync with the main file server every couple of hours, or every night using e.g. rsync. Now you have every data redundant in few places on two machines, and you can easily make manual backups on tapes, or CDRs, etc. from one of this machine.
You can use RAID 1 or 5 level arrays to be secured against hard disk failures, but it won't protect you if someone just deletes important files, so the periodical backups are still important with RAID arrays. Read the Software RAID HOWTO.
This is how I would do it, not counting on everyone making daily backups of their hard drives. I hope it will help you in securing your office data. The key ingredients: Samba and rsync.
You could also install rsync on the Windows machines (if there is rsync for Windows — I don't know) and set some Windows equivalent of cron job to update the backup version stored on the main server every hour and manually after clicking some "sync" icon, etc. Of course, There's More Than One Way To Do It.
Krótko: kady Erotomek
W pimiennictwie ma swój domek.