Slashdot Mirror
Remote Backup of Windows Boxes w/o Samba?
reezle asks: "I'm looking for a good (free as in beer) method to have my Debian server back up some remote Windows machines. There is no Samba in the mix; this is supposed to be a strictly secure internet thing. I've been toying with OpenSSH on the windows computers as a good tunnel, thought of simple tools like ntbackup initiated from a script on the Linux box, but not all of the pieces have come together yet. I need to have the Linux box make the connection, back up data (full and incremental backups) and have that backup data get back to the Linux machine in an encrypted format (across the wild internet). Has anyone done something like this?"
Install an SSH Server on the windows machine, use SHFS to mount the remote filesystem to a directory, then use rsync to copy it.
Your best bet, in a cheap and dirty solution, is to use SCP. Set up a cron job to securely copy the files you need. No muss, no fuss, no samba, and all encrypted.
Have cron create a .tgz backup regularly, and download it through scp on your Windows machine.
Can't you set up a vpn between the two pc's with OpenVPN, and use samba over the VPN?
Cygwin and rsync. That should do it.
-molo
Using your sig line to advertise for friends is lame.
Use the windows backup program to have windows make the backups then have Windows copy them to the debian box via scp.
Or.... put Samba on the Debian box, use port forwarding and the loopback adapter to create a tunnel to the samba box and have the windows backup program write to the samba share which is only listens on 127.0.1.1
setup on windows box:
Loopback IP: 172.168.254.1
Real IP: XX.XX.XX.XX
SSH port forward from Local 172.168.254.1:139 to remote 127.0.0.1:139
Create an account for each machine on the debian box.
Windows backs up to \\172.168.254.1\MACHINENAME
I've had fairly good experiences with the Unison product. It works similarly to rsync but with a few enhanced features. And I quote...
Anyway, you might give it a look...
As mentioned earlier here, there are numerous SSH/SCP implementations for Win32. Search around. They're somewhat hard to find, but there's quite a few (make sure you choose one in active development. There are quite a few abandoned projects with security holes and other bugs). Honestly, I don't remember what one I use on my windows machine :) As a word of advice, do not use a cygwin distribution. They're somewhat buggy and need to run inside cygwin. The 'native' servers which link against some cygwin libraries seem to be allright though.
There's also another easier option. But, it will cost you. Use a "real" backup program such as Retrospect which will do compression and encryption (very strong encryption if you desire) client side. More often then not, this is what big businesses use. You can then safely use smb, ftp, scp, whatever you wish
-- If you try to fail and succeed, which have you done? - Uli's moose
Why not set up a VPN and tunnel Samba through that? That should take care of your fear of insecurity with SMB. Piss easy + secure.
Not sure what software is available for windows, but there is FreeS/WAN for Linux.
Use netcat (nc)
A version exists for Windows (it's what the kiddee's use), but it can be used for legit purposes if done properly.
Don't think that a small group of dedicated individuals can't change the world. It's the only thing that ever has.
You can push the data using rsync. Load the Windows / cygwin version of rsync/ssh on the windows box. Then sync the whole box up to a debian directory over ssh. If you get fancy you can build rotation sets, etc. rsync is very efficient at moving only what changes, handling include/exclude lists, etc. Great for backing up lots of data.
Do a google search on "rsync windows backup".
The only downside is that rync will "kill" your upstream internet connection (it is that efficient). Fortunately, there is a --bwlimit option so you can back off a little.
If you are looking for a commercial hoster with all of the scripts pre-built we are happy to help (sorry for the AD), but it is pretty easy to roll this yourself.
Although the clients do not have built in support for encryption, according to the manual you can run the clients through stunnel to encrypt the traffic between the clients and the backup server. Future versions are supposed to support encryption built into the client.
you dont need cygwin. putty on the windows machine can create a tunnel to your debian machine. do what you like from there
NTBackup that comes with Windows can NOT backup all the Windows system drive, only part of it. Windows XP and 2000 (not Windows 98) have crippled file systems, apparently to implement copy protection.
NTBackup presumes that you are a peon whose time is worth nothing, and you don't mind loading all of your programs again. Some people restore a backup over a working Windows XP installation, but this is said by Microsoft technical support people to be unstable.
The ONLY way to back up a Windows XP and 2000 system drive is to do a sector by sector copy. See my Slashdot story and a discussion of this: Experiences w/ Drive Imaging Software?. The story gives a method of copying under Linux.
However, most sector by sector copies require that you have complete control over the drive.
The only solution I've found is Acronis TrueImage. It costs $50, and it works, although there are some small flakinesses. You can backup a Windows XP or 2000 system drive while Windows is running and being used. You can send the backup over a network to a local Linux machine.
To send the backups over the Internet, you would encrypt the Acronis files with GNU Privacy Guard first. There is a way to submit the password to GPG in a batch file. Since anyone who has access to the password has access to the backup files, this is not a security risk.
Google has more links to GPG.
I'm interested in hearing about any experiences anyone has with this.
Microsoft enthusiasts: Please don't disagree with the facts here, they have been verified many times by Microsoft articles and technical support people. Specifically, the NTFS file system is crippled, and NTBackup cannot back up the working system partition.
Legato's Networker Backup is a multi platform, Enterprise level backup solution. It will back up to tape and to disk archives. We use it on all of our production level systems. You should be able to grab a copy from here.. I'm not sure if it's "free" or not since our Purchasing dept. deals with software acquisition but i've not had any problems downloading copies of it.
If you don't care about enterprise features (and if it turns out you have to purchase it and don't want to), use cygwin/rsync like everyone else says.
Unfortunately, it doesn't look the least bit free (in any sense of the word...) Other than that, it's pretty wonderful stuff.
Take a look at rdiff-backup. I've been very impressed with it. From the website:
rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, and modification times. Also, rdiff-backup can operate in a bandwidth efficient manner over a pipe, like rsync. Thus you can use rdiff-backup and ssh to securely back a hard drive up to a remote location, and only the differences will be transmitted. Finally, rdiff-backup is easy to use and settings have sensical defaults.
I found it to be very fast and reliable. And yes, it aparently does work under Windows.
Oh you said Windows... There's always Cygwin.
Bacula has a binary to work with windows. Of course, there WILL be some system files that you'll not be able to back up while the thing is running, but the majority of windows files should be backed up flawlessly.
Don't know if it goes through the net encrypted, tough. The security I would use is the (not default, heh) hash pass"phrase", compression, and VPN.
eliphas
Well since you said free as in beer not free as in libre, and since you did not say how many machines you needed to back up exactly, I might as well suggest to you that you try Arkeia. The free version, "Arkeia Light," supports a server with a single tape drive (no changer robots) and up to 2 client machines. So you can use it to back up 3 machines -- If you use more than that or if you have a tape library, you gotta cough up for a license.
Plenty of the solutions people have posted about are fine and dandy hack jobs and while they probably will work -- they would take a lot of time and effort to implement. Arkeia is pretty easy to install on the client end - you basically just install the client and tell it the name of the server - the acutal backup is managed completely by the server. Clients are available for practically every OS under the sun.
Arkeia has served me pretty well over the last few years, so even with the cost (not very much for a comparable solution from any other vendor) you really do get a pretty decent product.
We use rsync to mirror (back up) about 200GB worth of data from a remote office via partial T1 line. Would work well enough over ssh too I suppose.
Our setup actually runs the rsync server on the windows machine. Main problem is that the default one with cygwin dies mysteriously for no apparent reason. Some googling found us a fixed version to use -- 18 months on, no issues.
"But actually trying to use m4 as a general-purpose langage would be deeply perverse" --ESR
We use cygwin rsync+ssh as for backup purposes and data exchange between remote offices. This is the best solution we've found so far. Each office runs rsync every minute. Data is transferred via Linux server.
There are drawbacks too. For example, you cannot store SSH key on mapped network drive. And do not use it with Windows 98, bash script and rsync will hang after 1-2 hours.
I dual-boot to linux in all the Windows PC (Every PC has linux boot) then use ssh/rsync to backup the whole Windows directory. Of course all windows partition must be FAT32 (even for WinXP) or something read/writable by linux. Then ssh reboot to Windows.
Nothing fancy to do on Windows. It takes 10 to 30 minutes to restore Windows to the way it was exactly when it was backed up, including XP. Without linux, it would have taken a whole day with all the patches, drivers, exact configuration, applications install, more patches, etc. etc.
As I said in my original post, all the facts in it have been reviewed many times by Microsoft technical support people.
I specifically said in my original post: "Some people restore a backup over a working Windows XP installation, but this is said by Microsoft technical support people to be unstable."
The SID must be changed anyway. That is an entirely separate issue.
I was told by a senior Microsoft technical support engineer last week, and I have been told before, that duplicate SIDs also confuse peer-to-peer networks.
XCOPY32.EXE can copy ALL files on a FAT32 partition. I stand by what I said. ALL the information in Windows XP and Windows 2000 is in files.
DriveImage, and all of PowerQuest, is now owned by Symantec, which, in my opinion, is legendary for bad technical support. Acronis is way better than DI.
An OS installation image is another term for a real backup.
Sysprep, even the Windows Server 2003 version, is buggy.
It amazes me how willingly people accept Microsoft's adversarial behavior.
It is highly configurable and easy to set up automatic backup routines and you can monitor operations
using a webbased interface. BackupPC also supports various transfer methods such as rsync, samba, etc.
and makes use of compression and pooling of files to save diskspace.
Of course, getting some scripts using rsync over ssh or something like that won't be that hard,
but anyway, I recommend you to check out BackupPC.
Dantz Retrospect lets you back up Linux boxes from a Windows or OS X backup server using its own client. They only support RedHat, but they do have a tar.gz that can probably be made to work with Debian. It's what I use to back up my firewall/email/web/everything server. It uses multicast for discovery, so if you put it onto a firewall, you'll need to set up multicast so that the packets are internal instead of on the default (Internet) route for the backup server to see the Linux box.
First, I create backup files. To do this, I use the Backup utility built into Windows (for better or worse). Of course, this assumes you have enough freespace on your Windows volumes to accomodate the backups...
On the server side, I set up an FTP daemon and an OpenSSH daemon. The FTP daemon is configured to only accept connections from localhost, so one must log in via SSH to transfer via FTP.
Then, I use a simple Win script to
This has worked fantastically well for me, and the only thing I didn't already have was PuTTY. It's also as secure as any other SSH activity (pretty darn secure), and if you make the SSH session log in as a restricted "backup" user, it becomes very difficult to use this system to escalate privileges on the server.
We may not imagine how our lives could be more frustrating and complex—but Congress can. – Cullen Hightower
I've put something similar in place which utilized the built in NTBackup as well as a combination of SSH/FTP. Scripting out NTBackup is easy enough; you just create the BKS file it'll use, backup using that file, and then get the file over to your linux system.
r \blah
/exclude to the dir/file line in the BKS.
/j "Helpful Description SystemState" /f \\server\backupdir\name_of_your_backup.bkf /n "Helpful Description SystemState" /d "Helpful Description SystemState" /v:no /r:no /l:s /m normal /rs:no /hc:off
/j "Helpful Description BunchaDirs" /f \\server\backupdir\name_of_your_backup.bkf /n "Helpful Description BunchaDirs" /d "Helpful Description BunchaDirs" /v:no /r:no /l:s /a /m incremental /rs:no /hc:off
To backup the systemstate (you need to do this on your DC's and Exchange boxes), you have to do it locally.
Instead of putting in a directory in your BKS, just put the word: SystemState
on its own line.
Your exchange directory store:
DS \\EXCHANGESERVERNAME
You exchange information store:
IS \\EXCHANGESERVERNAME
Directories and files:
c:\whereever\blah\blah\blah
c:\anotherdi
To exclude:
add a
Note: The BKS files are in a sort of unicode format (thanks Microsoft). I was using perl to create the files before every backup so I didn't have to depend on changing static files on each system being backed up. Its not quite unicode . . . open the file in a hex editor and check it out if you want to write them dynamically.
To kick off ntbackup at the commandline and use that BKS, something like this would work:
ntbackup backup "@\\server\backupdir\YOURSERVER_SystemState.bks"
Or for an incremental:
ntbackup backup "@\\server\backupdir\YOURSERVER_bunchadirs.bks"
I was planning on exclusively using SCP to transfer the files between the systems, but I ran into problems with the Win32 SSH client and server. Anytime I wanted to move a file bigger than 2G off the Win systems, I'd end up with a corrupt file. (So, I had to resort to using FTP in some cases).
Only semi related, If you are backing up systems at a datacenter and eventually sending them to a repository in the office (or vice versa) and sending them over a limited data line (we only had a couple T1's), you might want to look at the shaper app. I was able to limit the traffic heading over that T1 line after it was conglomerated on a datacenter server so that the users in the office could still do their work.
After adding a few hooks to the various scripts to have it spew its progress to a central server that I monitor for errors, I had a fairly scalable script based backup system using built in backups (NTBackup on 2k/Xp/2k3 and Tar). Oh, and Bzip2 is your friend on the windows systems. Getting around 2GB file size limitations was the biggest pain in the whole setup. Go through some good QA and check every backup that is created for the first week; it sucks finding out your files are corrupted when you need them . . .
Maybe try BackupPC, which I believe supports rsync and SSH on Windows (with cygwin).
Here's another good site on backing up Windows machines.
A lot of people think they know a lot about Microsoft Windows, and are mistaken.
I'd suggest using a combo of https and webdav on the Linux box and Karen's replicator on the Windows side to keep consistancy.
It's about as close as you can get to rsync on Windows without resorting to cygwin, plus it can run non-stop as a service.
Hope this helps.
Can I get an eye poke?
Dog House Forum
Or one thing I use, to keep incremental backups, is rdiff-backup.
Trolls lurk everywhere. Mod them down.
I've done this before, between two Windows computers! There's also an SSH implementation for Windows and you can grab all the files that aren't locked or protected on that machine. As for the registry and protected files, well, you use something like Ghost for that. Your backup should consist of only user data.
The leaders are Veritas's NetBackup and IBM's TSM. Not that I'm endorsing either...however, having used all three, I'd pick either NBU or TSM over Legato every time.
Advice: on VPS providers
You want duplicity. It'll run over SSH, and uses librsync. You get compressed backups, and compressed incremental changes. You can easily go back to the version of the file from a few days previously, or the latest version. It requires Python, librsync, and either a ssh or ftp server. You can make it work with cygwin, or with ActivePython + some other ssh server.
This post will enter the public domain 70 years after my death, unless Disney buys another extension.
Making NTBackup operate at the command line can be tricky. You can use the backup wizard to create the bkf file for Exchange, and then specify it from the command line.
.bat file I use for naming and managing backups without the use of any external utilities. It is pure batch language on Windows 2000.
/F "skip=4" %%A in ('dir s:\backup\*email.bkf /o-d /b') do del s:\backup\%%A /F "skip=4" %%A in ('dir s:\backup\*system.bkf /o-d /b') do del s:\backup\%%A
/M normal /F s:\backup\system.bkf
/F "tokens=2,3* delims=/ " %%i in ('date /t') do FOR /F "tokens=1,2 delims=: " %%l in ('time /t') do FOR /F "tokens=1-5 " %%n in ('@echo %%i %%j %%k%%l %%m') do rename s:\backup\system.bkf %%n-%%o-%%p-%%q-%%r-system.bkf
/a /v:no /r:no /rs:no /hc:off /m normal /j "exchange" /l:s /f "s:\backup\email.bkf"
/F "tokens=2,3* delims=/ " %%i in ('date /t') do FOR /F "tokens=1,2 delims=: " %%l in ('time /t') do FOR /F "tokens=1-5 " %%n in ('@echo %%i %%j %%k%%l %%m') do rename s:\backup\email.bkf %%n-%%o-%%p-%%q-%%r-email.bkf
Here is an example
Move the files to Linux any way you want.
-begin backup.bat-
rem Make room by deleting oldest files, leaving four newest
for
for
rem Back up the C drive
C:\WINNT\system32\ntbackup.exe backup systemstate c:
rem Grab the date and time, strip out extra spaces and illegal characters, insert dashes and rename latest backup file
FOR
rem Back up the Exchange database
C:\WINNT\system32\ntbackup.exe backup "@C:\Documents and Settings\Administrator\Local Settings\Application Data\Microsoft\Windows NT\NTBackup\data\exchange.bks"
rem Grab the date and time, strip out extra spaces and illegal characters, insert dashes and rename latest backup file
FOR
Save for the fact he's looking to pull the backups from Linux not push them from Windows.
So he does a
on the linux box and then runs whatever backup solution he likes on the linux box (against the mount point). You can't get much more "pull from linux" than that.-- MarkusQ
The leaders are Veritas's NetBackup and IBM's TSM. Not that I'm endorsing either...however, having used all three, I'd pick either NBU or TSM over Legato every time.
I agree that Legato is best avoided - in gereral, it leaves a trail of dead bodies behind it...
As far as cpommercial b/u software goes, BakBone's NetVault is a far better (both in functionality and price) option than either Veritas (good but $$$) or TSM (a true power tool, also $$$, but also requiring *way* too much arcane knowledge about its internals to run and effectively.)
"The future's good and the present is nothing to sneeze at." - Roblimo's last
I personally use Cygwin for various and obivious reasons. However, if you need SSH and cannot install Cygwin, for your various and obvious reasons, consider the alternatives.
PuTTY has several other utilities that come with it. For tunnelling, PLINK comes to mind - a cmdline util for establishing tunnels without a TTY ! It's small and can be unzipped alone from PuTTY's homepage.
If you cannot even use PuTTY & its utils. You could resort to Microsoft's Services for Unix. This is large install 135 Meg or so. (Or install it somewhere, copy out the bin & libs you need, and install them by hand.) If installed with Cygwin, it will break Cygwin (ENV issues, etc.)
SFU 3.5 comes with a full suite of unix utils, compilers, etc, including SSH. You can go grab source and recompile for Win2K or XP.
It DOES NOT come with any servers - no sshd, no X, etc. *They don't want to step on their partners territory* (i.e. get sued anymore than necessary)
SFU is a free download from MS. The Licenses prevent them from distributing *this* version of SFU. Their future versions built in to the OS are supposed to come from private source so the can distribute them.
rdiff-backup does what you want.
I have windows installers available:
http://sol1.net/~dave/backup
http://rdiff-backup.stanford.edu
it solves your problem exactly
I use it - get blat to email you the logs and you are sweet
I wonder what the pros and cons are of using a version control system for backup, such as subversion.