Slashdot Mirror
Ask Slashdot: Networked Back-Up/Wipe Process?
An anonymous reader writes "I am required to back up and wipe several hundred computers. Currently, this involves booting up each machine, running a backup script, turning the machine off, booting off a pendrive, and running some software that writes 0s to the drive several times. I was wondering if there was a faster solution. Like a server on an isolated network with a switch where I could just connect the computers up, turn them on and get the server to back up the data and wipe the drives." How would you go about automating this process?
Don’t know of any off the shelf software that does this, but should be easy to homebrew if you have the available skill set.
At the very simplest, you could probably build a custom livecd linux distro to automate the process after plugging in the machine and inserting the CD/pendrive. It’s not as complicated as it sounds if you base it off an existing livecd distro!
More complex, you could do PXE if the boxes are capable/configured for it (if not, probably more effort to change the bios settings than it would be to plug in the CD).
You’re probably content just with the backed up files, but I’ll also throw out there that I’ve found a very effective way to back up old machines/drives is to convert them into virtual disk files. Lets you boot up the old machine in a VM and poke around should the need arise. (disclaimer: I’m a dev not a sysadmin, so this is purely from “at home” experience).
Then don't automate it.
You could set up ghost and re-image the machines, that would at least give you a head start... may be able to do a low level secure format as well.
Perhaps a diskless boot (either Linux terminal server project or Lessdisks or something like that) and then run a script automatically to do whatever you want to do. Establish a client name based on the client's mac address. Use that to store the disk's contents remotely. If done right, you can have a room full of these machines doing this automatically at once.
Most Respectfully Yours Mark Allyn Bellingham, Washington
Well you can back them up using clonzilla however I've never used it that way before so I don't know exactly how you automate it....
but if you were to do that, you could then just write a bash script on the end which does the wipe with DD for you too. Job done :)
- http://www.milkme.co.uk
Nobody has demonstrated the ability to recover data after that outside of a carefully controlled lab.
Microsoft User State Migration Tool + Microsoft Deployment ToolKit + Sdelete http://technet.microsoft.com/en-us/sysinternals/bb897443
You should be able to backup the profile, load the OS and run a zeroing delete on all "empty space" on the drive.
EA David Gardner -"... but the consumers have proven that actually what they want is fun."
There are two commonly used techniques to the wipe process.
In Europe the preferred method is to fold the paper in half before wiping. In the US the preferred method is to scrunch up the paper in a ball before wiping.
Check whether the PCs you are wiping did a number one or a number two. Male PCs do not need wiping for a number 1.
"That's the way to do it" - Punch
As for a whole problem solution, I think you will need to do a bit of DIY. But just a note on the wipe process. Just writing 0 to the drive repeatedly will not ensure all the possibly sensitive data is non-recoverable, you really need to write random 1's and 0's at least 3 times to each bit of the drive. For that there is no better program than Derek's Boot And Nuke (DBAN) that I think is available as a liveCD and is available to several distros, including The Ultimate Boot CD (UBcd) and that may be a good place to start for a single boot backup, wipe solution. if you can write a shell script that can run from a pen drive while UBcd is in the CDbay.
Take the hard drives out of them, label them, and stick them in a closet.
I would look at FAI or kickstart. For FAI a pretty early hook to backup and wipe.... for kickstart a %pre script.
Of course, if you are working alone, and don't know how to configure DHCP/NFS etc.... it may take you a couple of days just to get the basic setup going, as they can be very finicky, but the quickstart guides out there should generally be able to get you going. If all goes well, you could be working on your scripting in a couple of hours, if not..... well....I hate troubleshooting NFS.. (and don't forget to check your IPTables setup if you are having trouble getting it working...amazing how much better NFS works when its packets are not being dropped.
Overall, I like FAI better than kickstart, but thats probably because I have used it less and those early stages (DHCP/NFS mount) are hard to troubleshoot with kickstart since stage2 (and thus a shell with which to troubleshoot) isn't available until that works.... though.... you probably don't have the same constraints I do and can just switch USB keys and boot off a fully functional system to test poke around.
"I opened my eyes, and everything went dark again"
I would pxe boot each computer with some flavor of linux, mount drive, backup, unmount, and shred drive You should be able to create a script that runs at end of bootup to automate this.
I wonder if you could configure a boot disk to image the machine onto a NAS, and then wipe it? Do you care about verifying the image?
How configurable is a Symantec Ghost boot disk? Is there a FOSS alternative to Symantec Ghost?
How would you tell the images apart. It's inevitable...Mary will need a copy of her "My Documents" folder from 2 years ago.
I once had to manage 30 PCs with Windows where users needed an easy way to quickly install one or another image on the machine. So I made that every computer had a small (few hundred Mb) partition at the beginning of disk with stripped down linux installation, instead of running init when linux started my script was executed where I had everything scripted (which image to transfer in which direction etc etc), and I just added necessary boot options to grub menu so that users can easly pick a task.
There isn't a whole lot to optimize in your process. Backups and wipes take time. One thing that could save you a step is to run the backup from the pen drive. That would allow you to script the entire process, such that you only need to boot off the pen drive, preferably have it cache itself into a ramdisk and start the script automatically, then move on to the next box. That would bring the whole process down to maybe 2 minutes per box.
Having ghosted a bazillion machines this way, it's monotonous but if you create 4-5 of those pen drives, you can do a bunch in parallel.
-Billco, Fnarg.com
I guess you can go away with the pendrive boot and use PXE or any supported network boot so that you have one OS image for "backup boot" and another OS image for "wipe boot"
you can use iptables and other mac-based packet requests to serve the backup image on the first boot and the wipe image on second boot. I believe it's completely feasible with almost any dd-wrt router with an usb storage plugged in.
Acronis or Ghost Enterprise can do this with every PC on a single network segment.
[RIAA] says its concern is artists. That's true, in just the sense that a cattle rancher is concerned about its cattle.
DBAN + PXE could do this for you:
http://users.telenet.be/mydotcom/howto/linux/dbanbootserver.htm
First write a script that will push a Wake-on-LAN packet to each machine on your list--assuming you're doing this in situ as opposed to bringing it in for the wipe. Then author a PXE script that first:
1) Checks to see if it is backed up, probably using a file on the backup server that you're using to select a list of the machines to backup and wipe.
2a) If it is NOT backed up, back it up with the product of your choice, whether it's Microsoft's USMT (I'm assuming you're doing this on Windows machines, you should see what you can get away with using SCCM), Norton Ghost, Acronis, etc. Once the script signals completion, have it reboot.
2b) If it IS backed up, PXE boot to DBAN (a tiny linux image) with the security level of your choice. I can get away with DoD standard here. Shut down after completion.
3) PXE boot to the new image you're replacing it with.
Test this. Test this thoroughly. Automating a data wipe can lead to some spectacular scenes of failure a la Fantasia.
Everyone else (anyone else?) will answer the automation question, but if you're ever done a PXE based linux install, you're about 99% of the way there.
The mystery I have, is where are you going to store "several hundred" drives worth of backups? And who or what is going to back up and maintain and store and recover the backups?
I'm guessing the best answer is open all the boxes, remove the drives, install new blank drives, all done? Given the cost of storage and admin time, this might even be the cheapest solution.
If this is a forensics issue, its a heck of a lot simpler legally to stuff THE drive in a evidence bag and buy a new one, rather than try to explain how your image is a true image crypto signed so it wasn't altered after it was signed, except how do you prove it wasn't altered before it was signed, blah blah blah.
Are you talking about backups where you only store relevant user "my documents" type data which might be practically nothing, or merely all files on a stereotypically mostly empty drive which would be at most a couple gigs, or a full bit for bit forensics dump of hundreds of 1 TB drives?
There's a big difference between "it all fits on a single USB attached consumer grade 1 TB drive" and "We're gonna need multiple racks of multimillion dollar NAS to hold all the images".
How valuable is the data? If it leaked would you lose PCI / CC / HIPPA / SOX stuff and its the end of the world or at least your corporation and job, or is it just a university computer lab and the most valuable/sensitive thing is a couple rickroll videos and some lolcats?
What do you intend to do, if anything, with the backups? The simplest / cheapest / most efficient way to store backups might involve just throwing the machines in a rented storage room. Climate controlled if possible. You can rent a heck of a lot of storage space for a long time for the cost of a couple hundred hours of admin time.
Finally whats your liability? If for example, one doesn't boot due to hard drive failure or whatever, are you shipping it to one of those $10K data recovery places, in other words you actually care, or if you lose some, eh, whatever, it was just a "nice to have"? If you can lose one, can you lose all of them with the same "eh" attitude? If your liability is significantly lower than your costs, your best plan might be to skip the backup and destroy the drives.
In summary the problem isn't how to "transfer" a couple hundred terabytes, that is a long solved question, no big deal. The unsolved problem is how to store / collate / search / backup / distribute / secure a couple hundred terabytes.
"Science flies us to the moon. Religion flies us into buildings." - Victor Stenger
1) Setup a private network for all the PCs you need to backup/clone
2) Setup a NFS or SMB share, PXE boot server and DHCP service on a Linux box
3) Create and add your backup/wipe script to the PXE boot image, have it execute at boot
Setting up a PXE boot server is very easy, first hit on google has straightforward instructions: http://linux-sxs.org/internet_serving/pxeboot.html
Most desktop systems have supported PXE booting for some time, you should be able to select it from the boot selection menu at start-up.
Live with the tedium of doing in manually. It sucks, but unless you are going to have to do this exact operation again in the future, don't bother with automating it. Possibly the solution of taking out the hard drive, putting in a drive dock on another computer, and letting that computer back-up and wipe the drive might be slightly less tedious, depending on the situation.
Because, if you listen to what you are asking, you are trying to set up an automated back-up and erase system. Unless you have a Lot Of Time to Test this BEFORE HAND, you could easily end up with an automated screw-up-the-back-up and nuke-everything system. If you successfully manage to create a system that erases several hundred computers without making usable back-ups, that might be a career-limiting move.
You are asking for replacing a single-shot pistol with a high-powered Gatling gun -- this is not unreasonable. However, if you shoot yourself in the foot with such a thing because you are not careful, there will not be a lot of remains left over.
If all the computers are absolutely identical, you might be able to do an automated system, test it against a couple machines, and be able to get it to work. Otherwise, the amount of time you will spend making sure that the automated system does exactly what you need it to do, safely, without ever failing, may end up being as much time as it takes to do it manually.
Oh, you are verifying that your back-ups are usable before nuking the drives, right?
If you're doing this for secure disposal, there's a much easier solution:
Pop the drives out and do your work via external slot-loading drive caddies. You can get rid of the big machines as usual and work your way through the drives as time permits between other tasks. If your software has command-line APIs, it should be pretty easy to setup scripts to do this.
- or -
Do the backup as a separate task. Deploy a dedicated backup tool (for de-duplication and compression) or use rsync. Then setup DHCP with NetBoot to run a distro configured to auto-format and install from an image hosted on a server. Even Windows supports this.
I'm out of my mind right now, but feel free to leave a message.....
Single server pxe boot into a live linux distro with clonezilla and your drive wiper of choice. Some simple scripting to get clonezilla to backup all drives to the server under the name gotten from a prompt and wipe when it's done. Throw the same bits on a USB drive if you want.
No sir I dont like it.
PXE booting is not difficult to set up and Clonezilla is dead simple to automate after that. DBAN also has instructions to PXE boot, but I've never used it that way. Extra points for setting it up to do both in 1 pass. Clonezilla also has the nice feature of verifying that you have a good backup.
That's perfect for the wipe, but he also needs backup.
PXE boot to pxelinux for a nice menu, then choose the backup (e.g. Clonezilla), throw a sticky note on the computer to indicate state, then reboot after the backup and choose DBAN.
http://www.linuxjournal.com/magazine/pxe-magic-flexible-network-booting-menus
A clever setup would keep track of which machine is in which state and hand out DHCP options accordingly, making the menu unnecessary.
With that many computers, backing up the drives to a ZFS volume with deduplication enabled is probably worthwhile (but make sure you have at least a GB of RAM per TB of disk).
My God, it's Full of Source!
OUTSIDE_IP=$(dig +short my.ip @outsideip.net)
For software backup, Norton Ghost enterprise is the way to go unless you have some solution you already are using / have to use...
For wiping the hard disks, they used to make bench-top hardware boxes you could hook up 4 drives to directly and mass-copy them all 4 at a crack. You can use a clean formatted drive as a source and "duplicate" that to wipe the drives clean, 4 at a time simultaneously.
I'm not sure if there are similar devices that do Ultra-ATA or SATA, but it might be worth looking into getting one if you have that many devices and time is critical.
-- You are in a maze of little, twisty passages, all different... --
Acronis will do this off the shelf.
If you are working with Windows Server 2008 R2 there is a way to automate all this as well as automate the djoin process using sysprep.xml. Depends on your level of knowledge I guess.
The easiest way, if you have a server with hot-plug drives, is to remove the drives from all the workstations/servers that need to be wiped and plug them into the hot-plug server (if you have enough carriers). Then you can run enough copies of your wipe utility to wipe them in parallel. We recently had to wipe about 400 drives, and we had two servers with 8 hot-plug SATA slots that we used to wipe them all over a couple of days. We were required to keep an audit log for the federal government of each drive wiped, so the software we used was modified to fetch the drive serial number and record that along with the time/date wiped. A modern chipset will support hot-plug SATA, so you don't even need to reboot the server. Just swap drives.
Fags will be fags, I guess. Do you even know why you want to write "zeros" to the drive more than once?
Set up a PXE server, where every device default boots the backup script.
Once that script is completed, that PC reboots, and the mac address for that PC gets shifted in the PXE config to a static the wipe boot script.
So have a watch process modify/update the PXE config for each machine as it transitions from backup script reboot (drops ping response?) to next PXE attempt.
How about:
1: Pop open case.
2: Remove Drive.
3: ???
4: Profit (through continued employment).
?
You only want 1 copy of the data, so the original is the backup.
Just convince the nearest PHB that it's a waste of your time and their money to wipe drives. If the machines are going to be repurposed or sold, it's cheaper and easier to buy new drives, or sell them without the drive. As for the backup, same deal - cheaper and easier to store a bunch of HDDs in anti static bags in a box somewhere. Safer than storing their data on a live backup device anyway. And if you need immediate, constant access to that data - why are you killing the machines in the first place?
But if you have to do it their way, get Acronis. It lets you backup an entire drive to a network location from a bootable cd / usb thingamajig.
It works with like, every fucking SATA / RAID controller there is, and if it doesn't, you get official support for building a BartPE disc that includes those drivers so you can get your shit on. You can even do a full drive backup from within windows. It's pretty fast, too. You can encrypt backups with AES-256, and you can browse through the backups with any machine that has Acronis installed. You could also of course dump the full contents of the backup into a 7zip archive or some other format if you wanted to not depend on Acronis's file format and software in the future.
Just make as many copies of the disc as you have network ports in whatever dark corner you'll be working in, line em all up, tell them to go, then play Minecraft until it's time to switch out machines. I think the bootable cd also has basic 0 wiping tools, as well, but I don't know for sure.
... where is /dev/one. we have /dev/zero but no /dev/one.
could be useful when erasing a drive with nothing but ones. dd if=/dev/one of=/dev/sdx
Three words...Ball Peen Hammer
Why not:
1) Remove disk
2) Put disk in safe as backup
3) Put new blank disk in machine
4) ?????
5) Profit!
That seems unwise. You're not really wiping the drive, just making it harder to read. Most modern wipe software overlays the drive 7 times with random data.
2 networks, one for clonezilla and the other for dban. /end
both booting via pxe with defaults set.
I'm pretty sure the prize money for recovering any data from a drive that has had only one parse of 0's is still up for grabs.
Why not roll a small Linux distro (something really titiny, like DSL, or picolinux, write a script to back up to thumb drive, then reboot the machine and wipe it? Push that image to every machine via network.
space is pretty cool.
That will make the backup a lot easier.
The first question that pops into my mind is, what is determining this secure wipe procedure, and how secure does it really need to be? If you're looking to speed things up, you could wipe everything with zeros once instead of "several times". The difference in security is minimal.
Aside from that, there are open source solutions that will image a drive and others that will wipe the disk. It shouldn't be too hard to chain them together, though I don't know of any pre-built solution. I'm stating the obvious here, but if you do try to build your own, definitely be careful of 2 main things: make sure users can't boot into this procedure accidentally, and make sure that the procedure absolutely will not wipe the drive unless the backup was successful.
Computrace by Absolute Software has a cloud-based end-of-life wipe (the product is called Computrace Data Protection). All the device needs is an Internet connection.
I'll go on record with a preemptory "what he/she said" to include all of the previous comments. This is one of those things that nearly every slash dotter has to do at one point or another and in my experience, you'll have to deal with at least one of three bottlenecks: time, money, or bandwidth.
If you are doing this to several hundred machines, sneaker net is likely a faster solution than your network. Take advantage of the higher bandwidth and save the backup images to a portable USB drive. With a large enough drive you can also keep the new image local as well. Using this method you can boot into Your Favorite Backup Solution, take the backup, securely erase the disk, and write the new image in one fell swoop. After the imaging is complete collect your various USB drives and march them back to the server.
With large enough USB disks, you could repeat this process several times before off-loading the images to your server.
Writing zero's IS wiping the drive. There is nothing else to read. "Modern" wipe software can overwrite 23929348 times, that doesn't mean it's actually doing anything neccessary.
If the machines are Linux (or booted temporarily into Linux), use ssh (or rsh) to script most of what you're doing. Be sure to configure them to not require passwords for ssh. Then use rsync to back up, and remote ssh scripting to do the wipe on all machines. You can get smart with transferring scripts to the machine & running them with ssh scripting without doing anything manual.
If the machines are Windows boxes, you might want to look at some remote access/backdoor solutions (of the "gray" hat variety, perhaps -- since you presumably do not want to go to each machine and log in manually to do ANYTHING). Do what the biologists do and turn attack vectors into something useful!
Boot off something, pxe or usb will do, a recordable cd or dvd if you must. Either have it give you a boot menu (grub works) with the options to run the various steps, or run one script fully automatically: Backup, verify backup, if it verifies log it and run the auto-wiper. If not, log it and call for attention.
Really, why do you have to ask? With a hundred boxes to go, yes it does pay to spend a day tooling and testing, then do the rest automatically and in parallel. Don't even have to wait for all the smart-ass commentary on slashdot. Get to work.
Ma Deuce, Bobcat.
Maybe a large bonfire in between.
What about writing a simple script that would do the same job.
1) Just have the script goto a computer
2) upload the data you need to a SCM hub of some sort or any kind of backup source.
3) Then have the script start a DD on the computer to wipe the drive, or a comparable windows program
4) Have the computer reboot and connect to a PXE server which has an auto seeded OS install on it.
5) Have your script push the information back to the computer
I wrote something remotely like this last year, it didn't do all these steps but it did handle remote access, backup and restoring of certain files. It was C based.
Most modern infrastructure management tools like Altiris can easily perform a pxe boot function. Set up a wipe job, link it to the MAC address and wipe it. Bonus points for having an auditable trail if that's required by your flavor of regulation.
of recovering any data after a successful single pass with
dd if=/dev/zero of=/dev/sda bs=4k
I'm just curious. I've read all the theoretical stuff, but wouldn't the drives have to be disassembled in a clean room and the platters installed on some machine that can read the faint magnetic residuals...
Who has these facilities and machines, if anyone, beyond the alphabetsoup gangs?
Doing things right isn't allows quick and easy.
FOG is a PXE cloning solution. http://www.fogproject.org/ Install FOG and storage where you want backups, setup PXE IP on network, and input all MAC addresses you want backed up. Through web interface to clone all. When done backing everything up, put a .img file of DBAN on the FOG server. http://www.dban.org/ Configure it in the FOG PXE boot menu, and make it an option but NOT default. Add appropriate start up flags for the level of wiping you want. Restart all computers you want to wipe, and select wipe option after PXE boot menu comes up.
I suggest you set that option with a password, since it will be available on all computers, not just the one's with the MAC address since only the FOG boot authenticates to MAC, not DBAN.
I8-D
If this is the only time you'll ever do it, a pen drive sounds good enough, although a CD image might be better since you could make a ton of them quickly.
Otherwise, piecing together a PXE solution would be a waste of time since you still have to plug the machines in, configure the BIOS for PXE, unlock the BIOS if you're planning on donating the machines (the bios steps can be done with automated utils if you're using HP or Dell machines).
If you can leave the machines where they are, and they're already unlocked and set to PXE boot, then making the PXE server only let their MACs attach would be the only big step. Then backup each machine to a folder based on its MAC.
Another option is to just take all the HDDs out, dump the machines, and backup/wipe at your leisure using a few computers with 6 drive cables each.
When I got data that I know is going to be over-written, I just assign a strong encryption-key to it. So, when I got a hard-drive with 2TB of data I flush the encryption-key and file-table to reset it. The encrypted data without the key is just noise, anyway. So, no hard deletes are necessary. This is a very secure way to work, and I am surprised not more people use this method. There is some software that can help you, but there are also scripts that can do the same job.
Why not just PXE boot a small OS image which does the backup and wipes the drives?? Linux or DOS will do the trick just fine. If you're trying to backup anything NT based (NT-Win7) get a read-only NTFS driver on the PXE image too. It's pretty simple really and doesn't take any real time IF all of your systems are semi-modern.
sigs are like a box of chocolates, they all suck remove the underscores to email me
ubcd has several utilities disk wiping and other disk management tools.
http://www.ultimatebootcd.com/
I say, you take off, and nuke the site from orbit.
Its the only way to be sure.
What I would do is configure a laptop to run DRBL or Windows Deployment Services (WDS). Both will give you PXE boot options and can boot whatever Linux (DRBL) or WinPE (WDS) utilities you want to use. WDS is a part of Windows Server 2008 R2 and for what you are going to need it for, you shouldn't have to purchase a license since the evaluation period should be sufficient time for you to complete your process. My suggestion would be to customize a Windows PE image to run a backup utility to capture all the data and write it to wherever you are putting it at, then run Gdisk32, which is a part of Norton Ghost to wipe the drive once the backup is complete and verified. You should be able to script this so it runs automatically once the PXE boot completes off a WDS server. I'm sure there's a way to do the same thing with DBAN if you're going to use DRBL instead.
THERMITE
i would like to see somebody recover ANYTHING from a drive that has been turned into a twisted molten blob
Any person using FTFY or editing my postings agrees to a US$50.00 charge
Always network boot first.
3 pxeboot configs; backup, wipe, localboot
2 corresponding tftp configs, which boot 2 different ramdisks.
First is a backup image using the tools of your choice. Last thing the backup does is write a flag to shared storage which tells the boot server to switch a particular machine to wipe mode.
Second boots the wipe image. When the wipe is complete, the pxeboot config switched to localboot.
Now you have a network of centrally managed systems you can manage by changing a couple of pxeboot files.
HTH.
Deleted
debian installer is very much misunderstood or at least underappreciated. i did a very very large (significant deviation / automated installation) system for automated customised installs of KDE desktop. some people said i would have been better off creating debian packages with postinst and preinst scripts, but i liked the convenience of being able to edit the shell scripts etc. etc. *without* having to run a debian package-create command. the results of the work are still here: http://lkcl.net/d-i/
anyway, you could quite easily use debian installer over PXE netboot - you just put the target repository that you want things to be loaded from into the standard kernel boot params (into the PXE config file) and debian installer will go "oh, i'm supposed to run in automated mode and to pick up scripts from http://lkcl.net/d-i let's do that then, eh?"
then you could have one automated installer which does the "wipe" process and another which does the "backup", and another which does the "reinstall". all specified via simple editing of PXE config files. just don't get them wrong, eh? :)
btw, in 2001 i was part of a team that did this sort of thing, entirely automated from a database. it was very cool. we even solved booting up and reinstalling NT systems (fast), by having a sysprep-enabled part-completed boot image that just needed finishing off.
How about removing the drives from the machines and doing more than one backup and wipe at a time? Linux dd doesn't have a problem doing the backups of anything as long as it is mounted, and wiping would be a lot faster and easier without all those reboots and hoops you have to jump through. That's how I would attack the problem. What sense does it make to boot and backup and reboot and wipe when the drives can be easily removed from the machines and wiped attached to a processing machine. Hell, you could write scripts to do it automated and come back after lunch and do the next set of drives.
Offer to pay people who need the work minimum wage to do it for you?
Not sure why most of the previous commenters thought you were redeploying these machines. Sounds like they are being surplused to me if you're writing 0s to the drives.
You can script as much of this as you want.
/mnt/backup //someserver/someshare /mnt/backup
/dev/sda /mnt/backup/someName-`date +%Y-%m-%d`.img /mnt/backup/someName-`date +%Y-%m-%d`.log
/mnt/backup/someName-someDate.img --- Note the sector size and start. Multiply these together. Example is 512(size)x2048(start)=1048576
/mnt/raw /mnt/backup/someName-someDate.img /mnt/raw /mnt/raw
1: boot a linux live image (CD, Thumbdrive, PXE)
2: mkdir
3: mount
Copy the raw device to the network share. We'll use ddrescue rather than DD so that it finishes even if the HDD has issues. You'll also get a nice log of the issues.
4: ddrescue
Wipe the disk
5: dd if=/dev/zero of=/dev/sda bs=1M
If you would like to see the data in the image
6: fdisk -l
Mount the raw image and take a peek
7: mkdir
8: mount -o ro,loop,offset=1048576
9: ls -l
Restore the data to the drive
10: dd if=/mnt/backup/someName-someDate.img of=/dev/sda bs=1M
You can then compress the image file to save some space on the server.
If you are microsoft based, the answer is in two solutions.
Free: Use the MS Deployment Toolkit and associated tools (WAIK, USMT, WDS, etc). It allows you to automate the deployment of any and all MS OS to various platforms, inject drivers, migrate data, repartition/format drives etc. Deployments can be based off of various deployment models including PIXE boots (with WDS), CDs, thumbdrives, etc.
Not Free: You might want to consider incorporating a System Center Configuration Manager (SCCM) environment. Then you can schedule and track your deployments across your domains automagically. The tools used in SCCM incorporate the same tools as the free solutions above, but the SCCM environment gives you more options and control over the deployment processes. SCCM will also give you a broad range of deployment options for software and configuration settings that you can control centrally.
Ghost and other similar tools can also do what you want to do. But for my money, SCCM is the way to go.
The system administrator Q&A site Server Fault at http://serverfault.com/ is probably a better forum for insight on this sort of thing.
What is the hard drive interfaces (IDE, USB, SATA, or ...)?
You could use hot plug-able hardware and software.
Never heard of PXE have you?
It is insanely easy to set up pxe booting if you're already running dhcpd. One line in a config file and you're most of the way there.
In the Windows world it's naturally more complicated but still easy if you know what to look for. If you have a recent Windows Server OS look into Windows Deployment Services and Windows Preinstallation Environment.
There are a bunch of good posts here, but most people are jumping to solution without knowing some pretty important requirements.
1) How important are the backups ? Is your system backup failure rate is 1% is that ok ? How many backups can fail, affects how much effort you need to put into the validation you need to do to make sure that you've got every backup. I'd suggest you should use whatever you use for regular backups, automate the agent installation and removal - this way you can plug into your regular corporate backup system - and avoid creating a whole load of bespoke process.
2) How important is it that you wipe 100% of the disks ? if your wipe failure rate is 1% is that ok ? (as above - if the answer is no, then you need to put the correct amount of effort into the validation that you've wiped the disk, and into creating an audit trail to prove the wiping was done).
3) Are there certain machines you really, really need to have copies of and make sure are wiped ? eg the CXO's ? If you make sure the really important ones are dealt with, can you accept a higher failure rate on the others ?
Easy to do, on boot it runs a backup of the drive, I.E. does a image of it, then when done dd /dev/zero > /dev/hda
reboot for each step? why?
Do not look at laser with remaining good eye.
Here's what we do where I work:
You'll need a Windows Server 2008 R2 with Windows Deployment Services role. You basically want to set up an isolated network with PXE booting, load a Windows PE disk into the PXE server. Modify the PE image to mount a drive off the server (to store your backups), then run a wipe script. As soon as the backup is done, you can actually fire up the next machine, you don't need to be 'connected' to wipe the disk.
For our purposes, we use Active Killdisk to wipe, and ImageX to backup.
You'll need PXE-enabled NICs or a gPXE boot disk. You can also do the exact same thing with a Samba server, a PXE daemon, and a basic Linux boot drive being served-up over PXE, but the learning curve is steeper. Also, ImageX is pretty awesome because it allows single-instance storage. You can append ALL you computer backups to one image file and only the different files will be stored more than once, cutting a massive amount of time and space from your backups.
"Sometimes, I think Trent just needs a cup of hot chocolate and a blankie." -Tori Amos on Nine Inch Nails
look into degaussing the hard drives with a dedicated unit. You can buy them anywhere from $2000 and up. They're usually good for hard drives and tapes as well and will erase everything including the factory servo tracks. This works regardless of the condition of the drive (bad controllers, no problem). It's good solution if you want accountability for audits, for example.
1) Get all the computers together in a room with a couple of monitors and a few keyboards. ... etc ... etc
2) Plug monitor+keyboard into computer 1, start backup script
3) Plug monitor+keyboard into computer 2, start backup script
4) Coffee break.
5) Plug monitor+keyboard into computer 1, start wipe
6) Plug monitor+keyboard into computer 2, start wipe
7) Coffee break.
I did something similar in the past, and ran everything off a boot floppy. I didn't even need to plug the monitor in, just boot from floppy, hit enter twice, "y", enter, and wait for the beep at the end of the process. It helped that all the computers were identical.
Although I do not know of anything that is completely automated FOG is pretty damn close. We have used it for a few years now to do exactly what you're asking. BUT as stated before, you do have to do touch the workstations... www.fogproject.org
When you overwrite data with a known set of data, it becomes much easier to recover. It's just a problem of signal analysis. Check out the Gutmann Method for more details on how this would work.
I cannot help you with a good solution, just a little advice. Verify that your backups actually hold data that you can access. It's good to see the data on the drive but can you access that data as well? I learned that in a 200 class I took at the school of hard knocks......
archive old drive, and send out system with blank drive.
Definitely PXE.
However do not forget the TIME it takes to backup and wipe. I hope your disks are not too big! If they are, you might want to partition them to use only a small part of the disk.
More to the point, what is the aim of your requirements? Maybe you would be better served by
- diskless workstations
- encrypted disks
- encrypted partition with the key in the boot partition (wipe the 1 MB partition containing the key and bingo you've wiped your 3TB disk)
Darik's boot and Nuke for wiping and FOG (Free Open Ghost) for taking and deploying images images. You would need to only visit the machine once to register with FOG. All the rest can be done from a web interface and I think you should be able to do all at once or at least in groups depending on your network strength.
1. Register with FOG
2. Take image with FOG during registration.
3. Load PXE image of Darik's Boot and Nuke (WIPE with Auto Nuke)
4. Deploy Sysprepped image!
5. Get a raise!
This has been covered several times here and elsewhere, but you don't need to write the 0s more than a single time.
Do you have ESP?
If the backups are "for the archives" and don't need to be easily searched or accessed, it may be cheaper to just replace the disks and store the old ones off-site with the rest of your long-term backups.
Knowledge is how to play a game, intelligence is how to win, wisdom is knowing what game to play.
Many of you are talking about opensource tools and although they are great!, I wanted to share with the community a tool that i found that has really made my backup/wipe administration duties very easy and secure.
http://www.9proof.com/prodotti/wiperaser/
I took this data straight from the website itself.... As i am at work right now and cant remember all the details.
9PROOF Wiperaser Ultra is the most comprehensive and innovative software for the secure deletion of data. Its primary goal is to permanently delete files and folders so that there is no chance of recovery, as also envisaged by Legislative Decree 196/03 on privacy, as well as various international standards. ...) you can delete data directly from Windows Explorer (Windows Explorer). The various features of integration with the operating system shell can be enabled or disabled according to policies defined by the system.
It supports every file system recognized by Windows operating systems (including FAT, FAT 32 and NTFS) and has specific functions to eradicate even the data contained in the basket and zombie-free hard disk space.
In addition Wiperaser is strongly integrated into the shell of Windows. By automatically adding an entry in the contextual menu (right click on files, folders, drives, trash
It 'also possible to create tasks , or sets of delete operations, which can then be performed on user demand or scheduled to run automatically by your company's information system.
Worthy of note can delete files in use or locked by the system using a technique called Direct Disk Access, which allows the ' direct access to the disk bypassing the constraints imposed by the operating system.
The function of remote administration and the possibility of imposing limits and constraints for each installation, makes 9PROOF Wiperaser Ultra particularly suitable for business use, allowing the adoption of policies for the secure deletion of information system controlled by the administrator.
The proper use of Wiperaser guarantees' can not be recovered later deleted data, protecting your privacy and ensuring compliance with the provisions of law; also features advanced security and management make it ideal for use in reality company making it possible to define exactly what users can actually permanently delete and what should be preserved.
Read more: http://www.9proof.com/prodotti/wiperaser/
FOG was created to have Ghost-like backup functionality using PXE and TFTP for network booting, and acquired a whole bunch of features in the process, such as disk wiping. From a clean web interface you can have FOG backup up the hard disk contents then use the "wipe" feature to erase the hard drive, quickly or thoroughly . FOG can handle as many client connections as your hardware can handle, but it's common to see a FOG server easily handle 10 or more client PCs at once.
- Locate the not too bright youth at hand
- Show it how to disassemble the drives
- Show it the slash hammer to destroy the drives with
- Explain management how you forgot to tell it about the backups...
- Done
If your server that is going to do the job queries its own arp table, will get info each time a machine requests an IP address, so you will have an unique identifier (The mac address). Then, you need to do some sort of backup and some sort of cleaning. I have no preference in the backup tool, but for the cleaning, you could use DBAN (http://www.dban.org/) it is a nice open source tool that will take it’s time, but will do a great job
You can write a script to SSH to each machine, copy whatever you need then wipe it. Wiping the machine remotely is the problem, there's a good article from Linux Journal by Kyle Rankin that describes a good way to do this. It's pretty recent (maybe two months ago), I haven't been able to find it on their website.
Centralisation is faster.
Just get all the machines to log into a server. Then wipe the server. It is much faster if the machines share a single common wiped drive than all the machines trying to have their own dedicated wiped drive.
I am anarch of all I survey.
Wouldn't a simple bash script off a live CD work well?
Assume:
1. The destination is known.
2. The software environment is known (same boot CD across the board).
Process: /dev/sda
1. Calculate MD5 for
2. Determine the MAC address for the computer
3. Mount the share location on the remote server
4. Use dd to image the drive, and pipe it through gzip, creating on the server
5. Add to the mac-address.md5 to the gzip file
6. Server auto-verifies this via a cron job that runs on it (pulls file out, checks MD5, compares, red-flags any that don't match)
Unless you're looking to use the drives after the fact...
Wow. Im amazed at these comments. Has nobody ever thought of SCCM or Altiris? Duh.
Dear slashdot. Tell me how to do something I should have e\learned to do when I went to school for Network development. Maybe people will tell me how to for it for free. This reminds me of the free loaders in a couple of my classes. How can i do x? Do you have some code I can look at? Can you explain it to me? How can I make it do Y?
i guess the main issue is going to be broadcast domains , are you on a single subnet ? if so a PXE debian client boot with a shel lscript will do the trick , if you are running multiple subnets , this can be anything from easy to hellish , if you are getting the harddrives physically in your hand i would look at wipping them electromagnetically .... (are you living near cern ? or any other electro magnet powerfull enough to rip the filling from your teeth ?
BACKUP:
-Buy a Server with a substantial volume of disk space.
-Server runs a PXE boot program (Windows Server has an entire system for deploying boot images to machines; many flavors of linux do too)
-Server is shotgunned via 8, 1gbps lan links to a 24-port cisco switch. This guarantee's enough BW for every station to be running full-bore.
-Server has either a SAN with 16 disks or has 16 disks built into it. HP can sell you such a beast for under 10k; dell for under 6.
Plug the machine in, boot, configure for pxe boot, that grabs the boot image from the server and actuates it. The boot image starts up WinPE, which in turn maps the server to a local drive, names a share after the unit PN/SN/AT whatever, then starts up norton ghost utility OR a Foresnics imaging utility and dumps the data onto the server.
The server, then, from oldest to newest, moves the images onto tape or an online storage service for safe keeping.
WIPE
-2nd room has old P4 machine.
-Pxe Server
-Boots up an image of darik's boot and nuke.
-It has a shell where all you need to do is hit the enter key to start the nuke; this way you don't accidently nuke anything.
All you need from there is Paperwork to CYA for mistakes. I'd recommend stickers and a REALLY REALLY good pre-copy process whereas someone gets the machine in and puts all the data into the computer via bar-code scanner.
After backing up get a big magnet and wave it over the hdd a few times. That will clean it very well!!!