Microsoft Releases Public Beta of Data Protection

Torrey Clark writes "Microsoft has released the public beta of its disk-to-disk backup product, Data Protection Manager. The product is designed to make backups easier than simply backing up to tape. Disk-to-disk backup completes images in significantly less time, meaning much less downtime for systems during backups."

Comments

downtime during backup?

[R.D.Olivaw]

Don't know about the rest of the world but we don't have to take systems down to backup them here.

Re:downtime during backup?

[cduffy]

Agreed -- though snapshotting *is* necessary to get consistant state, that's provided by AFS (when backing up our fileservers), Oracle (when backing up database contents) and LVM (everywhere else).

Re:downtime during backup?

[Anonymous Coward]

obviously you're not backing up disk snaps of windows drives.....

Re:downtime during backup?

[FullMetalAlchemist]

You don't with Windows either, but you have to make sure there are no handles to critical files when you do. After that you can just use dd or whatever, I use dd, because I came from a unix background and found it the most simple solution.

If you're not in the know and still reboots, why not just g4u?

Re:downtime during backup?

[ntshma]

Microsoft Data Protection Instructions: 1: Click on START and then SHUTDOWN.

Re:downtime during backup?

[abysmanu]

Sometimes systems will need to be brought down before performing a backup. Otherwise the image might be in a state of flux, causing potential problems during recovery.

Some backup utilities provide capability to take a snapshot and backup that snapshot while the system continues to be used.

One of the features Microsoft is touting in this product is 'moving only the byte-level changes of the file servers' thus eliminating any downtime.

Re:downtime during backup?

[Anonymous Coward]

Well, on linux and unix, it's trivial, as most enterprise-grade systems are configured with an LVM,which allows snapshotting of a consistent filesystem state when combined with a filesystem such as XFS. On linux you just use xfs_freeze to quiesce the filesystem, lvcreate a snapshot volume (large enough to hold changes that will happen to the main system during the backup - the paranoid use a snapshot volume the size of the main volume, but I usually get away with 20% or so with a wide margin), xfs_unfreeze, and backup the snapshot volume. No application interruption, just minor performance impact.

Re:downtime during backup?

[Anonymous Coward]

Some backup utilities provide capability to take a snapshot and backup that snapshot while the system continues to be used.

Yes indeed, and this is built in to modern linux FOR FREE (just use LVM snapshotting and XFS). You can even avail of it in windows, if you squint a bit: if the windows filesystem in question is really a samba share (obviously that won't work for windows system filesystem (at least in part due to windows' obnoxious file system locking semantics) but for user homes and application data, it sure can - we do it!). This is another example of linux+samba being a better windows file server in some ways than windows itself.

Re:downtime during backup?

[Gillious]

I think what they are getting at here is the time it takes to restore the data. Rather than taking the time to restore from a tape which is very time consuming, it images to a hard drive. That drive can simply just be plugged in when the machine is down for the drive replacement. As opposed to taking down the system, replacing the drive, reloading the OS, installing the backup software, restoring the data, verifying it's integrity...

Re:downtime during backup?

[TummyX]

xfs_freeze might make sure that the fs level of the files is fine but what happens if it occurs at a time when an application is doing something vital. I doubt all applications on linux are written to be xfs_freeze aware.

Re:downtime during backup?

[ASkGNet]

I would imagine that xfs_freeze just redirects all write i/o to a snapshot volume and overrides all read i/o with data in the snapshot volume (if the data requested has been written since the execution)

The applications don't even know what happened, it all happens at VFS level

Re:downtime during backup?

[Ozric]

Why dont you set your tablespaces to backmode and do a hotbackup. You dont need to take it down.... need some Arch logs tho. Then there is always RMAN.....

Or write a snapshot to a 3rd mirrror.

Sounds to me like MSFT just found out about tarballing to an NFS mount point.

Re:downtime during backup?

[operagost]

I've been able to do image backups of disks in OpenVMS since -- who knows when. Maybe since it came out in 1977.

MOUNT DKB0: /FOREIGN
BACKUP /IMAGE /IGNORE=(INTERLOCK,NOBACKUP) DKA0: DKB):

Congratulations, you can now boot from DKB0: Or save the image to a saveset on tape or disk instead of overwriting a FOREIGN-mounted disk. Same program. Slap a fancy GUI on that and you have MS DPM.

Re:downtime during backup?

[ajs]

This is why you use a NetApp. Backups are atomic, guaranteeing filesystem integrity, so recovering from a restore is exactly identical to recovering from a system crash, except you have more guarantees about the state of your data. I've used NetApp snapshots (the fundamental building block upon which backups are based on a NetApp) to back up Oracle databases that were under heavy read/write load, and restored backups of same. No worries.

Re:downtime during backup?

[TummyX]

That doesn't Change anything.

How do you know your backup was made at a point where the file data is stable frOm an apps point of view? Either the app has to be made freeze aware or the app has to be closed.

The snapshot volume you mentioned helps to keep the sever running but any data you backup would be like data copied from a drive from a machine that was powered off without shutting down. You never know just what wasn't in a stable state! Maybe apps need to be written to be backup aware.

Re:downtime during backup?

[Taladar]

MSFT discovered Raid 1?

Re:downtime during backup?

[Dego]

Probably because you are only backing up simple files and directories. Try backing up SQL Servers that way.

Re:downtime during backup?

[basingwerk]

Or use RMAN/NetBack integration to make a full hot backup of your data using a 12 terrabyte tape robot and a backup catalog without stopping operations - that's what I call backup.

Re:downtime during backup?

[ekarjala]

OpenVMS was around in 1977? I mean, it is great that VMS had a rich feature set, but how does that help those of us that manage enterprise environments today? Since the original context of this thread was backup on the Windows server platform, how about something really useful. Check out an OpenVMS based product from PolyServe that actually works on a current day, pertinent OS's with real market share.

Re:downtime during backup?

[FreeUser]

Don't know about the rest of the world but we don't have to take systems down to backup them here.

Yeah, it's pretty pathetic. Contrast their approach with our simple "poor man's RAID" backup solution which has worked on Sun systems, *BSD systems, and GNU/Linux systems for over 10 years:

(install two identical hard drives)

dd if=/dev/hda of=/dev/hdb bs=1048576


Run as frequently as you need a backup image. This has worked, as I said, for over a decade, and has allowed quick and easy recovery of every machine we've ever had a hard drive crash on. There are theoretical limitations, e.g. a gradual failure of the source drive that causes corrupt data to overwrite the backup, so "poor man's RAID" should only be one component of one's backup strategy, not the entire strategy, but it has worked wonderfully. Why not use RAID instead? Well, in some limited cases we do (including dual RAIDS that act as "poor man's RAID" to each other), but the problem with RAID alone is that you can't necessarily snatch the drive out of one machine and put it in another (hard drives aren't the only things in a computer that break) and because a RAID isn't a backup, it's a highly available drive, which means when you need to recover older data after, for example, human error damages the current copy, RAID does you no good while "poor man's RAID" gives you an instantly accessible copy of last night's backup.

That, coupled with more traditional backup to tape, or, as we've been doing lately, a tar archive of the entire system's filesystem to a central RAID repository across the net, is pretty damn bullet proof, easy, and trivial to recover with.

Re:downtime during backup?

[jonored]

Or, you know, using a proper file server to serve data to windows machines... where your backups consist of the occasional 'tar -c ' to a tape drive, and if you catch the problem early enough you just tell it to roll back to a snapshot.
And if they are... 'tar -x'.
And then the restore on the windows machines goes something like "insert Knoppix CD, boot" then "dd /dev/hda". Or perhaps setup a new windows machine. Windows machines seem to not make as much sense in general when dealing with them.

Re:downtime during backup?

[jonored]

dd, tar, etc. on a *nix platform - and Apache has something like 66% market share.

Re:downtime during backup?

[Anonymous Coward]

not exactly, but close. xfs_freeze stops the filesystem in-kernel process writing changes to disk (it blocks on writes, so to the application it's just like the disk is slow), lvm snapshot creates a snapshot volume, xfs_unfreeze starts the filesystem writing again. Any changes to the main filesystem go to the main filesystem, the snapshot records how the main filesystem used to be at time of snapshot - i.e. when a block is erased on the main filesystem, it's added to the snapshot. The snapshot is a backward diff of the main filsystem. This is more efficient and reliable for slightly obscure reasons left as an exercise to the reader, but note that it's important to free the snapshot once you're finished with it or things stay a little slower than they should be.

But in essence, yes, the applications never know it's happened.

Re:downtime during backup?

[Anonymous Coward]

Well, in theory you're kinda right, but in practice all enterprise-grade applications I know of are coded defensively due to the already existing power-off without shutting down problem and thus recover adequately, particularly since any good programmer will know that any vital state data should be written synchronously.

There are OSes that shoot for full OS-level "orthogonal persistence", but no mass-market mainstream OS, windows or unixoid, has the feature in full generality, so all large apps I can think of look after their own safe persistence (a significant programming overhead, but one that is pretty much a Solved Problem that only total muppets (and I admit the windows and recently linux (due to expanding non-guru usage) worlds are full of such muppets) get wrong.)

Re:downtime during backup?

[lgw]

The problem with what you just said is that some systemes don't keep disk in a constistant state. This cannot be fixed with anything in the filesystem. If an application only uses disk to keep its state between shutdown and startup, and keeps everything in memory otherwise, you clearly need to back up that application through an API.

MS's new product doesnt do that except for MS products, of course, but real backup software does.

Re:downtime during backup?

[lgw]

The real problem with this approach is that it's not a backup, it's a copy. This is usefule, don't get me wrong, but you have no mechanism to move the data to tape.

Why would you want to use tape when disk is so cheap and easy you ask? If you're backing up your desktop, no reason. If you're protecting corporate data, you *must* have offsite backups for disaster recovery and (often) regulatory compliance.

This new MS product, combined with a good backup product, would be great, except of course there will be no integration. But wait a while and I'm sure backup companies will be offering the same sort of product integrated with real backup.

Re:downtime during backup?

[lgw]

That, coupled with more traditional backup to tape, or, as we've been doing lately, a tar archive of the entire system's filesystem to a central RAID repository across the net, is pretty damn bullet proof, easy, and trivial to recover with.

That's the key. Otherwise it's just mirroring, which isn't much protection. Over 80% of restore requests come after the user says "oops", less than 20% come from hardware failures (thanks to RAID). If you can't give your COE back yesterdays version of the spreadsheet or powerpoint presentation he just saved/deleted/corrupted/whatever, you don't have much job safety, even if your hardware is bulletproof.

Re:downtime during backup?

[mcrbids]

For offsite backups, rsync is your friend. I wrote a nifty wrapper for rsync (in PHP) to manage backups of multiple systems and accounts with options so that unchanged copies of a file only hardlink to a single file.

I manage about 500 GB of data in backups over a 1.5 Mb line with this tool. I can "rollback"
files on the servers to any of a number of points in the past several months, since this tool does versioning. It works very, very well, and I have a complete, updated, off-site image of all the servers I manage every 24 hours!

Re:downtime during backup?

[AssHatAnonymous]

You've never worked in an enterprise enviroment have you? Tar-ing files that are in use just doesn't work. You get inconsistent data. Come back when you've had some real work experience or at least get a little cleverer than 'tar'.

Re:downtime during backup?

[soulhuntre]

"You don't with Windows either, but you have to make sure there are no handles to critical files when you do."

There are several XP and 2K3 backup products that don;t need to worry about this. They can backup the system on the fly without worrying about open handles and so on using the API's supplied for this purpose.

Re:downtime during backup?

[Anonymous Coward]

Well, you really should put the DB in hot-backup mode first, but the principal is sound. We use EMCs in the same way; sync the BCV (business continuity volume; essentially it's a mirror of the live data), put into hot-backup mod, split the BCV, remove from hot-backup mode. Then mount the BCVs somewhere else (we use our DR servers; others use the backup node itself) and back up the data.

We have used the BCVs to restore a database and that worked like a charm.

If you don't use hot-backup mode, you essentially have data in the same state as if the server crashed at the point you split the BCV/make the snapshot.

Re:downtime during backup?

[RichMeatyTaste]

Exchange servers stores have to pause for backups. We've got smaller stores (6-7 gigs) and they take about an hour to backup to our older DLT library. Disk to disk would reduce that quite a bit.

Re:downtime during backup?

[ishepherd]

Some backup utilities provide capability to take a snapshot and backup that snapshot while the system continues to be used.

Yep, even the free Backup utility with Windows XP has that ability. It's very handy too.

Possibly it was in Windows 2000 also... I forget.

Re:downtime during backup?

[operagost]

The dozen credit unions who invested in new Alpha server systems from my company in the last year (and the half-dozen currently on my department's plate) would be horrified to learn that a random slashdotter thinks OpenVMS is outmoded.

We'll also be moving to Itanium this year, running both Windows and OpenVMS in hard partitions.

Re:downtime during backup?

[Anonymous Coward]

Then why didn't the VMS guy (David Cutler ?) have this implemented in WNT? He had full power, all Bill asked him to do was 'make a Windows server'.

One more thing...

[teh_mykel]

Now, if Microsoft could actually release a product that didnt require an amazing array of backup software, we'd be talking business.

Re:One more thing...

[Ziviyr]

I just use dd, since any file level backing up in a live Windows can't do a complete job, and writing NTFS via FOSS has never stabilized (scary commentary on their filesystem).

Re:One more thing...

[Anonymous Coward]

What, you want Microsoft to get into the business of making unbreakable hard drives?

Let's start off with small miracles first.

Re:One more thing...

[Harassed]

I just use dd, since any file level backing up in a live Windows can't do a complete job

Yes it can and has been able to for some time, at least on the server side. Windows Server 2003 has a service called Volume Shadow Copy which is designed to do exactly that.

Re:One more thing...

[Ziviyr]

As a non-server user a few years before 2003, I used to adore

dd if=/dev/hda1 | lzop >foo.lzo

Re:One more thing...

[ErroneousBee]

Ive been using partimage, as it drops the unused parts of the filesystem. Never had to do a restore but did test one once.

Im about to start using subversion for backups of my home directory, and syncing my laptop with my desktop.

Re:One more thing...

[Ziviyr]

Unused parts of the filesystem compress well after pretreating with the results of

dd if=/dev/zero bs=1G count=blah | bzip2 -9 >foo.bz2

(7zip decompress what you need, make a few zerofiles of varying size, dd rules)

Re:One more thing...

[moonbender]

Are unused (unallocated) parts of the filesystem read as zero by dd, though? I kind of doubt that. Disallocating (say, by deleting a fie) typically doesn't zero out blocks, it just updates the allocation table to disregard the contents, right?

Re:One more thing...

[Anonymous Coward]

Yes it can and has been able to for some time, at least on the server side. Windows Server 2003 has a service called Volume Shadow Copy which is designed to do exactly that.

Shh..they prefer to stay ignorant and spread Fud. Because in their limited experience, they have never encountered it, so it must not exist.

Re:One more thing...

[Ziviyr]

dd doesn't give a flying poo about what data is going through it, so its not analyzing BAMs and such.

My point was, fill the filesystem to the brim with 0x00 files, wipe them out, then pipe dd, through a compressor (which will catch the zeroing done, bzip2 works really well here), then to where its going.

Re:One more thing...

[pegr]

My point was, fill the filesystem to the brim with 0x00 files, wipe them out, then pipe dd, through a compressor (which will catch the zeroing done, bzip2 works really well here), then to where its going.

With the added benefit of overwriting unallocated space. You don't want deleted files to be recoverable*.

(*Unless you do, of course...)

Re:One more thing...

[timster]

bzip2 on a very large block of zeroes is fairly processor-intensive, so might not be appropriate for all situations.

gzip compresses zeroes much faster, but not nearly as effectively.

Re:One more thing...

[lgw]

since any file level backing up in a live Windows can't do a complete job

This is more true than people realize. There are *dozens* of services in NT that can't be backup up as flat files. However, any good backup program will handle this. Even the built-in backup utility will handle this (I used to think of that thing as tar with a GUI, but it does actually work for those annoying services).

Re:One more thing...

[Ziviyr]

I also once suggested reading straight to lzop.

I'm not sure bzip2 starts operating unusually slow on zero entropy data though...

Surprising

[ThePlague]

The really surprising thing is that they released the source code, and here it is:

xcopy *.* "x:\" /d/s/e/c/f/h/k/y

Re:Surprising

[Flagg0204]

I've had Microsoft's Disk backup utility since DOS 5.0. diskcopy

Re:Surprising

[Lil-Bondy]

"format x:" -- sourcecode of the many viruses to soon hit microsofts brand new backup product

Re:Surprising

[Uptown Joe]

Only they will call it Data Protection Manager 2006 and charge $800 (Base XCopy) for it... Plus the $400 Exchange Agent (/EA) and the $400 SQL Agent (/SQLA) and the $400 Multi-Server (/MS)Agent and the $400 Netware Compatibility Agent (/FU) and the $40,000 Linux Compatibility Agent (/HAHA)

- usage: Xcopy c:\*.* x:\ /h/e/c/k/HAHA...

Re:Surprising

[noidentity]

The really surprising thing is that they released the source code, and here it is:

xcopy *.* "x:\" /d/s/e/c/f/h/k/y

Sir, a DMCA takedown notice has been filed with your ISP and Slashdot. Please remove all source codes at once.

- Friendly protector of your rights

Re:Surprising

[ConceptJunkie]

Yet they managed to make a 700MB installation file out of it!

Now given that this is a sophisticated server backup tool that supports SQL Server, Windows 2003 Server and all kind of other stuff, I'm sure it's a bit more than just "xcopy ..." (of course, I use "robocopy /MIR" myself), but 700 MB?!

StoreAge Networking Technologies support

[xtracto]

So it seems DPM is only a "data-mover", so it will need to be combined with another technology, after some research i found this:

StoreAge Networking Technologies announced that it will be developing enhanced solutions to support Microsoft System Center Data Protection Manager

The full article is: here

Rsync works fine for us

[Tanami]

We've been doing disk-to-disk for a year or so now using rsync's --link-dest feature to create apparently complete mirrors each night, but with only those changed files actually occupying disk space (beyond that of a symlink). Makes restoration an absolute breeze compared to tape, but I'm not sure if this M/S effort does the same? *runs off to look*

Re:Rsync works fine for us

[gtoomey]

I rsync from my web server in USA to Australia using my ADSL connection.

Rsync usually reports a 1000 fold speedup over a dumb copy.

Re:Rsync works fine for us

[Colin Smith]

Which is great till you start thinking about stuff like disaster recovery, offsite copies etc.

Tape's fine if you have decent software managing it. My personal recommendation is Tivoli Storage Manager if you have money to throw at it and if you don't then Bacula. More are however a little more than simple network backup systems and may be overkill on a small network.

Re:Rsync works fine for us

[Tanami]

Which is great till you start thinking about stuff like disaster recovery, offsite copies etc.

I don't understand? That's precisely why we started doing it.

We have an offsite server in a managed facility to which we back up each night over SDSL - nightly update via RSync for ~400GB of total data is around an hour on average. This server collects data from two sites, in the event of total system failure at either site, we've got lots of options depending on the disaster - home users could connect directly, somebody could physically go and pick the box up and install it as a replacement, etc, etc.

Much less effort and downtime than any tape backup system we've ever encountered.

For day-to-day restores (deleted files, etc) we have a similar system at each office, which implements the symlinked backups - I could restore right now, in literally 10 seconds, any file on our network shares exactly as it was at the end of any working day between now and the 5th of January. Perhaps more importantly, pretty much any competent user can do the same, without having to bother me. Show me a tape system which will do that!

Re:Rsync works fine for us

[Anonymous Coward]

1000 fold speedup....wow.

That's about 1 followed by about 300 zero times faster.

1000 times speedup would be Speed x 1000, 10 fold speedup would be Speed x 2^10.

Re:Rsync works fine for us

[Colin Smith]

" nightly update via RSync for ~400GB of total data is around an hour on average"

Ah. That explains it. 400Gb is what I'd describe as a small system. We do an incremental of around 16Tb and we're not a particularly large site. Pushing 400Gb over a WAN is expensive enough, try it with a bit more data.

"I could restore right now, in literally 10 seconds, any file on our network shares exactly as it was at the end of any working day between now and the 5th of January. Perhaps more importantly, pretty much any competent user can do the same, without having to bother me. Show me a tape system which will do that!"

Already mentioned it:
http://www-306.ibm.com/software/tivoli/produc ts/st orage-mgr/

Re:Rsync works fine for us

[zeromemory]

rsync is a great backup method, but there are some problems when trying to use it on a Windows server. cwRsync (rsync for Windows) isn't able to backup locked files.

This presents a problem because many users are accustomed to leaving programs and files open during the night, when the backups are performed. Apparently some third-party vendors have come up with ways to backup locked files, but I don't know of any open-source tools that do.

Re:Rsync works fine for us

[Diag]

I could restore right now, in literally 10 seconds.... Show me a tape system which will do that!

Well perhaps not 10 seconds, but 25 seconds. A high end tape library can mount a tape in 11 seconds, the fastest tape drives can load and locate in 11 seconds, and then actually read the data off the tape, at up to 60 MB/sec ultimately.

Obviously this is the high end stuff that costs big bucks.

I'm not saying there's not a place for disk in backup, but in high volume shops, tape is really still the best option for long term data storage.

Re:Rsync works fine for us

[Taladar]

What exactly do you do at a "not particularly large site" that generates 16 TB of changes every day? Definitely not the typical office usage.

Re:Rsync works fine for us

[Tanami]

(and in reply to Colin as well)

Okay, I think you guys are talking about an order of magnitude more data (and probably two orders in cost?) than your typical business that just need to backup some project data and possibly emails.

I would imagine it's the latter group that MS are going after, rather than people with serious quantities of data.

Re:Rsync works fine for us

[Tanami]

Just noticed something - it's even smaller than you think - 400GB isn't the diff, that's the total size of the data - the diffs are probably a couple hundred meg at most.

I still think we're closer to the "average" business in requirements than your 16Tb ;-)

Re:Rsync works fine for us

[Tanami]

We found that, we also stuggled (failed?) to find a version which supported large files properly.

Fortunately, the bulk of our data is on Linux machines and accessed via Samba anyway.

Re:Rsync works fine for us

[lgw]

You've described a good solution for someone with quite small data protection requirements. Just be aware that you're not capturing the state of the Windows install, as the files on disk for a running system are pretty meaningless (there are dozens of services each with it's own API for backup). However, if you're not talking about a database or mail server, you can probably get away with this, and just re-install Windows before restoring the actuall work-related data when a machine is lost.

A tape drive that holds 1TB per tape is pretty much wasted on soemone with 400GB of data. :)

Re:Rsync works fine for us

[ManxStef]

Robocopy's a good alternative to rsync if you're after basic capabilities and only running Windows (and don't need compatibility with an rsync server). It's in the Resource Kit - 99% sure that the 2003 RK version is backwards-compatible so go for that one (as you can download the 2003 RK rather than having to order a CD for the 2000 one). More here:
http://www.microsoft.com/downloads/details.aspx?Fa milyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&displa ylang=en
http://www.ss64.com/nt/robocopy.html
http://www.windowsitpro.com/Articles/Print.cfm?Art icleID=39119

What ever happened to easy backups?

[pcmanjon]

What's wrong with:

dd if=/dev/hdb1 of=/mnt/hdh1/path/to/desired/backup/image/here.iso

Oh, not available for Windows, so you'll have to buy a product instead. But isn't dd much easier than using a program that expires after 270 days.

http://www.microsoft.com/windowsserversystem/dpm /e valuation/faq.mspx
Q. When does the DPM beta expire?
A. The Data Protection Manager software expires 270 days after installation.

Re:What ever happened to easy backups?

[Anonymous Coward]

Beta program expires afther 270 days. moron!

Re:What ever happened to easy backups?

[Anonymous Coward]

y helo thar linux troll

Re:What ever happened to easy backups?

[michich]

What's wrong with: dd if=/dev/hdb1 of=/mnt/hdh1/path/to/desired/backup/image/here.iso

The fact that you call the resulting file *.iso even when it's highly improbable that it contains an ISO9660 filesystem.

Re:What ever happened to easy backups?

[maxwell demon]

What's wrong with:

dd if=/dev/hdb1 of=/mnt/hdh1/path/to/desired/backup/image/here.iso

Depends. It won't work if /dev/hdb1 is your largest file system. And of course you don't want to do that while /dev/hdb1 is in use, since you otherwise risk inconsistent data structures.

Re:What ever happened to easy backups?

[xchino]

"Oh, not available for Windows, so you'll have to buy a product instead. But isn't dd much easier than using a program that expires after 270 days."

Check here, and after your done, go buy yourself a clue before you post utter bullshit again.

Re:What ever happened to easy backups?

[pcmanjon]

Yeah, but IIRC you have to reboot the system and such with this backup program.

Also, this program won't copy a 120gb partition to a 110gb partition either.

There's no way to shrink data like that without taking up a lot of CPU.

Re:What ever happened to easy backups?

[bersl2]

You have some factual errors in your post, but we get the idea.

I for one find it extremely humorous every time I hear some Windows user come along and ask for good imaging software.

Re:What ever happened to easy backups?

[frankblack9999]

Funny how people complain (above) about having to buy a separate product to enhance Windows, AND complain about Microsoft crushing the little guy when they bundle similar utility programs with the o/s.

Re:What ever happened to easy backups?

[Vo0k]

- After disk crash, rebuild -exactly- the same partition table to have your new hdb1 fit the backup of the old one.
- Back up all the empty space...
- at least mount /dev/hdb1 -o ro,remount if you want to avoid SERIOUS inconsistencies (in many cases it means downtime of certain services)
- except trouble with recovering to a different drive. Whether you back up hdb1 or hdb, the problems are different but present in both cases.

dd is very crude and sucks at backups. Good only if you want to "try something" and then recover the system to the same drive, same partition. Not for preventing disk crash, surface damage etc. A good backup software should be able to survive read errors to recover whatever possible from already damaged disk - dd dies on read errors.

I'd rather go with LVM/RAID1.

Re:What ever happened to easy backups?

[maxwell demon]

Unless you have your partition almost completely full (or the backup partition already contains a significant amount of data), it's certainly possible to backup a 120GB partition on a 110GB one without taking up a lot of CPU: Store only the used sectors.

Re:What ever happened to easy backups?

[Rurik]

dd conv=sync,noerror

'noerror' will not die on rear errors.

Re:What ever happened to easy backups?

[rwyoder]

This is the actual backup program that I run weekly on the SPARC machine I am using this moment: #!/usr/bin/sh ROOTPART=/dev/dsk/c1t1d0s0 BACKPART=/dev/dsk/c1t2d0s0 BACKPARTRAW=/dev/rdsk/c1t2d0s0 DD=/usr/bin/dd BLOCKSIZE=16777216 FSCK=/usr/sbin/fsck ${DD} if=${ROOTPART} of=${BACKPART} bs=${BLOCKSIZE} ${FSCK} -y ${BACKPARTRAW} All that is required to make the 2nd drive bootable is to edit it's /etc/vfstab and do s/c1t1/c1t2/g.

Re:What ever happened to easy backups?

[Taladar]

Might be related to those people not being the same people.

Re:What ever happened to easy backups?

[frankblack9999]

Neither of us know that. The point is, damned if they do, damned if they don't. When you have both sides of an argument angry at you, you're probably doing something right.

Re:What ever happened to easy backups?

[Anonymous Coward]

Yeah but the Windows imaging softwares aren't as picky about drive geometry as the *nix-wares are.
Try dd'ing a 80GB drive to a 120GB drive using dd, I dare you!

Re:What ever happened to easy backups?

[Saeger]

So what's the problem? I did that just the other day; after dd'ing I used 'parted' to resize the single ntfs partition to fil bigger new drive.

Re:What ever happened to easy backups?

[jacksonj04]

Standard MS procedure is that beta software expires. The anti-spyware beta expired, as did alphas and betas of Windows 2000 and Windows XP which I have.

Also, my Longhorn tech release has an expiry date (Can't remember what tho, and can't be bothered reinstalling to find out).

I think this is just to stop people from being foolish and leaving beta software on their systems when there's a newer version out.

Re:What ever happened to easy backups?

[smallguy78]

is that a deliberate Murloc (WoW) impression?

Re:What ever happened to easy backups?

[asdfghjklqwertyuiop]

This is the actual backup program that I run weekly on the SPARC machine I am using this moment:

#!/usr/bin/sh
ROOTPART=/dev/dsk/c1t1d0s0
BACKP ART=/dev/dsk/c1t2d0s0
BACKPARTRAW=/dev/rdsk/c1t2d 0s0
DD=/usr/bin/dd
BLOCKSIZE=16777216
FSCK=/usr /sbin/fsck
${DD} if=${ROOTPART} of=${BACKPART} bs=${BLOCKSIZE}
${FSCK} -y ${BACKPARTRAW}

All that is required to make the 2nd drive bootable is to edit it's /etc/vfstab and do s/c1t1/c1t2/g.

You run that without snapshotting? That's scary. You have an good chance of having corrputed files in the backup.

Re:What ever happened to easy backups?

[runderwo]

Complaining about bundling is nothing more than a warning for third party developers who want to choose Windows as their primary target platform - Microsoft may buy you out, or clone your product to put you out of business.

Now product tying is a completely different thing. Fortunately few of Microsoft's utilities are "tied" to Windows in the sense that you can't check them off in the installation. There are a few like IE and Media Player, and not being able to uninstall those without resorting to third party software is a legitimate complaint.

Re:What ever happened to easy backups?

[runderwo]

dd_rescue does not die on read errors.

Re:Useless backup

[Anonymous Coward]

aww if you had said platter-to-platter i would have agreed with you

But....

[Anonymous Coward]

Microsoft has also said that it won't be using its own software since it prefers to destory any information that could be used against it in a court of law.

Now you can

[R.Caley]

have your viruses and trajans back and working again in a fraction of the time.

Nerver more will you have to endure those painful minutes between rebuilding your system and getting re-infected.

Re:Now you can

[khujifig]

Oh, my spyware handles that for me!

Re:Now you can

[biggerboy]

How did a Duke basketball player from Alaska get on your computer?

Re:Now you can

[R.Caley]

I thought everyone knew that a trajan is a piece of malware which placates the user with pointers to cheap online food retailers and violent computer games while exploiting their network connection to attack PCs in Germany and the Middle East.

This must be another Microsoft first....

[suman28]

Imagine that...less down time. Who would have ever thunk it.

I dont understand.

[Anonymous Coward]

I RTFA.

Disk to disk backup would require the system to be shutdown, drive added, removed and reboot, configure etc. How this is better than Tape Backup? In fact Tape Backups do not require downtime at all.

Unless they are talking about removable media like CD/DVD/USB devices, this does not make sense. But in that case, this method would be useless for data larger than maximum allowed space on these devices.

Re:I dont understand.

[jmcmunn]

Thank you...you show me the hard drive that holds as much as my tape drive does, then we'll talk. And I certainly don't have room in the drive tower for another 7 drives.

Re:I dont understand.

[nojayuk]

Disk to disk backup would require the system to be shutdown, drive added, removed and reboot, configure etc.

Unless they are talking about removable media like CD/DVD/USB devices, this does not make sense.

There are such things as USB hard drives which appear as volumes within the Windows OS, you know. I use such a unit with a cheapo exchangeable ATAPI cartridge bay hacked into it for backup. I call it "tepid-swap"; it's not true hotswap as I have to "stop" the USB device before switching it off but I certainly don't have to shut down the entire system to change the drive fitted in the bay. After I power it up again Windows 2000 automatically recognises the new disk, no reboot necessary.

Re:I dont understand.

[0xdeaddead]

http://www.planetprices.co.uk/hardware/1058/210176 6/1/Any-400GB-7200-rpm-Hard-Drive.html 400 gig disks. How big is your tape drive again?

Re:I dont understand.

[Anonymous Coward]

What tape drive do you have? Our major problem right now is that our RAIDed hard drive sizes (now at 3TB and 3 x 1.6 TB on critical systems) have hugely outpaced our tape drive sizes, even with intelligent (i.e. only backing up the important stuff) and compressed backups. The expensive Ultrium LTO whatever tapes in a robot library that have a vague hope of backing up our systems are unworkably slow at actually doing so, so our tape system sits mostly idle. It's now mostly used for transferring large data sets from remote systems by the time honoured high-bandwidth, terrible latency "car full of tapes" method.

Our backup solution is another two arrays of hard drives that we rsync an LVM snapshot volume to nightly and weekly. So we don't have much point-in-time recovery ability at all, which sucks.

Re:I dont understand.

[nojayuk]

Thank you...you show me the hard drive that holds as much as my tape drive does, then we'll talk.

Well tell us how big your tape drive is and we can discuss it. There's a LaCie USB hard drive case available with a 1.6Tb capacity in one portable box, configurable as a single volume; do you have a tape cartridge that big? As a USB device it can be plugged and unplugged in a running system with a minimal amount of operator intervention and no need to power down and reboot the system. My own HD cartridge USB device allows me to swap multiple 120Gb hard drives in and out of my system similarly without rebooting, and I trust those drives a lot more than I would trust tapes, having been bitten by "bad-tape no. 3 of 5" syndrome before when restoring backups.

Re:I dont understand.

[Colin Smith]

1.3 TB. Also spools at a sustained rate of around 78MB/s so you basically also need a network interface for each drive unless you stick it on a SAN.

e.g.
http://news.sel.sony.com/pressrelease/3334

And you can throw the tape in a box and take it off site in case there's a disaster, which we all know is waiting to happen.

Re:I dont understand.

[0xdeaddead]

500 gig native.. with 38MB a sec... I think Id bench that vs Lacie 1.6 tb disk...
http://www.lacie.com/products/product.htm?pid=1055 1
The really scarry part is data retention.. Most US companies need to keep 7 years of backups, but have you ever tried to restore a 7 year old tape? Its like impossible.

Re:I dont understand.

[inject_hotmail.com]

There's a LaCie USB hard drive case available with a 1.6Tb capacity

You wrote a little b, is that bits??? Ok, I'll assume bytes.

Though I agree with your idea as I am the last person on the planet to be a tape-fanboy; however, imagine the how long it would take to transfer 1.6TB via USB:

1 600 000MB / USB's theoretical max of ~50MB/S = 32000s

= ~9 hours at top speed. In the real world, it's considerably less...

And then there's the added wondertude of USB's perfect compatibility and functionality record. :S

Usually ways other than tape or USB are my fav. I recently jumped on to the network storage bandwagon. I think it's great because if one has more than one network storage device configured properly, one can easily:

A) swap out the 'destination' drive daily to provide a rolling backup which can be EASILY transported off-site, and

B) Span the data to be backed up across as many devices as can be afforded (current cost is ~$2/GB) -- adding up to an infinite amount of backup space.

No, I don't work for Linksys or BuffaloTech.

Inject

Re:I dont understand.

[mabinogi]

that's 500GB. 1.3GB is the compressed value.
Unless you're storing a bunch of text you're not going to get 1.3GB.

Also - 500GB SAIT tape - $200
400GB Hard Drive - $320

That would make it look like the tape has an advantage - but how about you tell us how much you payed for your SAIT drive?

Re:I dont understand.

[mabinogi]

bah!, I previewed twice and still didn't catch it!

I meant 1.3TB of course ;)

Re:I dont understand.

[Colin Smith]

A SAIT cartridge is about 10cm square and weighs what? 200 grams? *That's* portable. The drive is capable of 78MB/s sustained, 50% faster than USB 2.

Course, you've got to pay for performance.

Hard drives have a place in backup, but it isn't for very high capacities or offsite storage.

Re:I dont understand.

[Colin Smith]

An average of 2.6:1 compression is perfectly reasonable, we get that on normal files, on our RDBMS files we get far higher, data rates of 78MB/s or better and 1.3TB capacity are easily atainable.

I've restored many 7 year old tapes. It's perfectly possible. Take it you leave yours out in the sun to ferment?

Re:I dont understand.

[Bombcar]

420LTO 2 drive

200 GB Uncompressed. Exact same size as a CDROM. Multiple tapes available.

How much does it cost you to have 30 hard drives? 30 tapes?

Redundancy, my friend.

Re:I dont understand.

[Colin Smith]

Binaries, word docs, text, database files all compress well on to tape. 1.3TB is about the average of what we get onto the cartridges. RDBMS files and engineering data sets in particular compress to far higher densities, 5TB -> 10TB per cartridge. The drive does the compression so it doesn't impact the client systems too badly.

You can have an external SAIT drive for around £2500. Ours are in big (Hundreds of TB) libraries and cost a bit more. They are actually physically a lot smaller than they used to be.

The point is that hard disk backups are for small networks. People who say tape is dead, back up to hard disk RAID arrays are people who back up small systems or sites.

Re:I dont understand.

[Taladar]

Compression is most likely the same independent from the medium used.

Re:I dont understand.

[Diag]

Restoring from old backup tapes is not a problem as long as you keep the hardware and software available, or make sure you migrate your data when you upgrade your backup system. Admittedly that's easier said than done, but I don't see that it would be too different whether your backup data is on disk or tape.

How many hard drives have you kept running for 7 years? How about a bigger company that might need to keep tens of thousands of drives running for 7 years, and pay to power and maintain them, just in case the data needs to be restored?

A tape in a vault can work out a lot cheaper.

Re:I dont understand.

[Colin Smith]

Sure, but with tape, the drive performs the compression, not the client or server. This means that backups during the day for instance have a rather lower impact on client responsiveness to the user.

Attempting the same thing having a client system perform the compression will significantly affect either the client's performance or the backup performance from that client. I would only ever have a client system perform compression if the network is otherwise inadequate. Having the backup server perform the compression severely limits the performance of the backup system as a whole.

For this reason, making use of compression on a tape drive is standard practice, and why compression is not standard practice when not backing up using tape drives.

Re:I dont understand.

[Frogbert]

Isn't SATA hotswapable?

Re:I dont understand.

[lgw]

Tape still has the cost advantage if you keep backups around (which many companies are legally required to). I suspect that disk will eventually be cheaper than tape *media*, however, just a matter of time.

The problem with disk, however, is it doesn't meet many requirements for disaster recovery planning and regulatory compliance. Tape is really growing strong with meeting compliance requirements, with WORM tape showing up from many vendors - disk may be cheap, but write-once optical (of the sort you need for multiple TB) is darned expensive.

Disk also hasn't proven itself for offline disaster recovery (the most expensive and best disaster recovery, where you have duplicate data centers in different countries is all disk based, of course). DIsk drives don't seem that reliable yet when you hand them to Iron Mountain and ask for them back in 7 years. Tape is proven solid for that quite common requirement.

Re:I dont understand.

[0xdeaddead]

Ok, now try going to a new company that has a 'legacy' tape drawer. The dont even know if it was netware, nt, unix or vms that made the tapes. Not to mention they had 2 sets and constantly went back & forth on them.. guess what? plenty of bad reads on those tapes....

Bottom line is tapes suck.

Am I missing something?

[(trb001)]

Or is this just RAID-1 backup without the read performance boost?

--trb

Re:Am I missing something?

[maotx]

Our company just takes out a scsi harddrive from our RAID1 and BAM, instant backup. Replace harddrive removed with another one and mirror. If their is a problem with the filesystem (say after a bad patch) out with the new and in with the old to go back right where you were. Of course this only works with our OS and not company data as it's stored on RAID5 and archived to tape.

Re:Am I missing something?

[jasonaedwards]

Just curious, as we are looking at a similar strategy, but what happens when the server hardware dies, and you need to replace with new and different hardware? The OS won't boot if the hardware is considerably different... you will probably end up with a BSOD, right?

Re:Am I missing something?

[ctr2sprt]

It could be useful for other things. For example, doing tape backups over the network. A full backup of a 50GB drive over the network is going to take two hours (or longer). During that time your server's hard disk is going to be thrashing like crazy and performance will be shot to hell. But if you've got two disks, then you just "break" the array (making one of the disks readonly) and backup from just one disk. Minimal performance hit, you get the benefits of RAID1 (and snapshots), and it's a pretty simple system so there's not too much that can go wrong. We have a couple hundred servers at work that use this approach and it's amazingly convenient and problem-free. A lot less flaky than most RAID hardware too, sad to say. (I'm looking in your direction, Mylex and LSI.)

Re:Am I missing something?

[MagPulse]

I just want to add my recommendation for RAID-1. I set it up a couple of years ago when my last drive died, and I decided I was sick enough of losing data that it was worth it. I just had one of the drives die on me last weekend, and my downtime was just the few hours it took me to go to the store, get a new drive, and swap it in. Then I just told my Intel RAID app to mirror to the new drive and I could continue working while it rebuilt the array.

Without RAID-1 I'd be pissed for the rest of the month. Seriously, who has time to do manual backups of your personal PC? And even then you'll probably lose something because it's not up-to-the-minute.

Re:Am I missing something?

[Anonymous Coward]

My god. If you need to ask this let your boss know he needs to take away your access to the server room.

Re:Am I missing something?

[lgw]

Someone mod parent up please!

This used to be a top-dollar solution, but there are seevral reasonable ways to do this now. If MS's new product were only integrated with backup software, it might be good for this as well.

A few points

[erroneus]

Someone criticized the "downtime" thing. Frankly, in order to get a good backup, any other processes running on data should not be in flux or the backup itself could be corrupt. So even in most conventional backup schemes, there is a period of time in which backups run and nothing else does.

Another point is that I do not see where it will support operating systems other that Windows. This is to be expected, but a mature solution should be capable of backing up multiple operating systems as many sites I have seen have a heterogenus computing environment. At my site there are Windows servers but there are also Novell, Linux and SunOS. Is there a solution for those too?

On the other hand, if we're talking about what essentially amounts to "dd" I am sure there could be a handy Knoppix CD created to suit the task in some automated way. It could actually be quite simple in that at a certain time of day (night?) power to a bootable external CD drive is enabled, the system is scheduled to reboot at the same time, it boots from CD, runs "dd" per the scripting in the custom Knoppix where it finishes the job by writing out information to a log file about success or failure and then reboots the computer again. That's just off the top of my head but I am sure that even more elegant schemes could be cooked up. This solution would be effective at creating viable images at a good speed and could even utilize compression along the way.

If Microsoft wants to make a "ghost" backup, then maybe they should just license the technology from Symantec.

Re:A few points

[RollingThunder]

You've never heard of using broken mirrors, I take it?

Zero downtime. Instant backup for a change. Sure, you need RAID-1 for it, but disk is cheap compared to the data on it.

Also, various products have quite capable open file managers. We use Veritas Netbackup at my workplace, and it's excellent, cross platform and high performance.

Re:A few points

[jbarr]

I have used Acronis' backup product on my workstations, and it works on-the-fly. I even tested it by doing a full backup, formatting the system disk, and restoring, and everything "came back" like it was before I did the backtup. Acronis certainly has an excellent product.

Re:A few points

[jabuzz]

I take it that you have never heard of snapshots and file system freezes then? It goes like this; freeze filesystem, take snapshot, unfreeze filesystem. Typically this takes place in under 10 seconds. Then you backup using the snapshot which can take as long as you need. Provided you don't run out of snapshot space of course. Then you release the snapshot once the backup is complete. Try man xfs_freeze for information on how you backup on real operating systems.

Re:A few points

[fr0dicus]

A nice idea until Murphy's law strikes.

Re:A few points

[lisaparratt]

That depends on filing system - I vaugely remember hearing something about XFS being about to log subsequent writes so that the image is of the drive as it was when the backup started.

Re:A few points

[Stephen Samuel]

...broken mirrors...
A nice idea until Murphy's law strikes.

Not an issue. There are a couple of solutioons:

1. 3 way mirror ... Break off the third mirror, back it up, then re=attach it before the next backup.
2. Primary is raid-5. Secondary is a simple concatonation (I presume Windows can handle this sort of setup.) Once again, break off secondary to do backup.
3. Filesystem (such as Veritas) which allows snapshot filesystems. Snapshot systems keep track of old data when new data is written to the filesystem. This allows you to have a static snapshot of the filesystem (at least, until your snapshot reserve gets filled).
4. Bulletproof server (oh hold on -- that's SPAM, not RAID).

Note that 3-way mirrors are an especially good idea for read-mostly filesystems, since extra mirrors will increase read bandwidth -- but decrease write bandwidth (unless done with hardware support).

Re:A few points

[marcosdumay]

Someone criticized the "downtime" thing. Frankly, in order to get a good backup, any other processes running on data should not be in flux or the backup itself could be corrupt. So even in most conventional backup schemes, there is a period of time in which backups run and nothing else does.

It may be a concern while you backup databases, but for usual files there is no corrupt state. I don't know how Windows stores its configurations, so, I don't know if it can have a trustable backup tool without downtime. If it uses a database, there is a big problem there to fix.

Another point is that I do not see where it will support operating systems other that Windows.

Even if it does, nobody will use, because the UNIX world alread have tools for that that are better (no downtime) and simpler. Some people who don't know those tools often create a fast, reliable backup system without downtime and that saves disk space with 2 or 3 lines of code. So nobody have the need for this backup system. Also, if you take the time to set Samba, you script can backup Windows machines without modifications, assuming that Windows doesn't store its configurations on a database.

Re:A few points

[lgw]

Frankly, in order to get a good backup, any other processes running on data should not be in flux or the backup itself could be corrupt. So even in most conventional backup schemes, there is a period of time in which backups run and nothing else does.

This is a problem with "broken mirror" style backups (as much of the stuff you want to back up is in memory), but not with backup software. Any good backup software will backup live applications using the APIs provided by those applications for that purpose. Databases, mail services, all those little Windows services, all have APIs for backup.

Now, this may still put an unacceptable load on your system, and you need a better answer, but mirroring alone isn't it. You need backup software that can cause Oracle or Exchange or whatever to sync itself to disk, then hold it inactive while you break the mirror (hopefully just a few seconds).

MSs new product is for "flat files" only I believe, so it doesn't really play here.

Re:A few points

[bergeron76]

At my site there are Windows servers but there are also Novell, Linux and SunOS. Is there a solution for those too?

Yep. It's called Partimage.

Re:A few points

[RollingThunder]

True, but you're talking about a very limited window.

The usual place for a broken mirror backup is in an OS upgrade, I've found.

You break the mirror, apply the patches to one half, and test. If things went down shit creek, you re-sync with the old mirror as primary. If it's fine, you re-sync with the new mirror as primary.

That's an outage window of about two hours.

In addition, you also use full mksysb's or OS level backups so you can still do a bare metal restore if things went really screwball... but since you never proceed with any warnings extant from EITHER part of the mirror, that's exceedingly rare, and it beats the crap out of taking the entire system down for a naked backup.

Ummm.

[Colin Smith]

Haven't RTFA because I couldn't be arsed.

Shouldn't the system back up to a disk spool and then to tape for offsite storage? Hell, even the freebies Amanda and Bacula do that already. And Yup, Bacula is available for Windows.

It does have to be said though that some very expensive commercial backup systems are only just managing to include disk spooling prior to tape ( Having had to deal with it for several years, I refer to that steaming pile of dung which is Netbackup).

Re:Expiring Backup Software?

[ozbon]

At risk of stating the obvious, it's a beta. Not a full release.

As such, yes it'd be nice if they gave it out with a "full life", but nine months is far better than 30day trial periods etc.

Any sys-admin installing a beta with a defined 270 day limit on it and using it in a "potentially mission critical environment" deserves to be sacked anyway.

Re:Expiring Backup Software?

[JaredOfEuropa]

That is, in my opinion, extremely dangerous.

It is a public beta, for evaluation. You're supposed to use this alongside your existing backup mechanism, in a 'sandbox' environment, or in a carefully monitored environment, where you should expect and prepare for issues with this product

Any sysadmin using expiring public beta software for production backups, shouldn't be a sysadmin in the first place. Don't blame Microsoft for this.

Why is this news?

[Anonymous Coward]

I don't recall /. articles for the release of any of these applications:

Freshmeat Backup Apps

(flame away)

Re:Expiring Backup Software?

[kahei]

Did you miss the word 'beta' -- or would you actually use a time-limited beta version of a backup system on a real server? I think you save more face by picking the former.

MOD UP PARENT

[Anonymous Coward]

Congratulations, Anonymous. You have taken the first step towards breaking free from the mat - er, /. group think®. I can only show you the path, its up to you to follow it. Now, which will it be, red or blue?

Re:Expiring Backup Software?

[GIL_Dude]

So you actually think we have admins out there so lame that they will ignore all of the warnings and their supposed training, install this in a production environment, and then not upgrade to the production code when it ships?

If so, they need to be fired and get a real admin. This stuff is for test labs only at this point. Anyone placing it in production now deserves what they get.

if data is corrupted

[bro1]

Now it would be nice to get 5$ each time data is corrupted by this backup system.

Re:if data is corrupted

[Anonymous Coward]

I'll settle for $100/hr to repair it. Maybe just $80...

It can be a bit more complicated than that

[WARM3CH]

Disk-to-disk backup? In fact, I (ie. my computer!) do it every night. Simple copy command? I think that does not cut it. I'm in a tight development cycle and each day write a lot of code, documents and receieve/generate lots of data files. I need to back up all important data but surely I don't need to make backup of the executable files, temp files, OS system files and such. The solution that I use is simple: I have two hard disks in my computer. The files that I need to have back-up from, are scattered on these two drives. Now, I have made a BackUP directory on each one of these drives and put a copy of all important files in them. So, I have 3 copies of every important files: the original, and two back-ups. In case a hard-disk goes banana, I always have a copy of all important files on the other one. I run the back-up every night. Just need to copy the files that has been changed or the whole new directories made during the day. So the problem is: I need two desinations for each source. I need to be able to select which directories or even which files to back-up (or not to backup) and I need to check which files have been changed or which new files (or directories) have been created. I need to be able to schedule the back ups for midnight and I need to forget about all these details in practice as I have to focus on my work :) How I did it? Well, I tried a script in the beginning but found it difficult to manage over the time and it was very tedius. Now I use SyncBack which is a freeware program with all these features that I need (and more! like FTP and compression to Zip, etc.). QED.

Re:It can be a bit more complicated than that

[tuxforever]

Yeah and I bet that doesn't have any spyware attached to it or anything...

Re:It can be a bit more complicated than that

[jasonaedwards]

not that I have ever detected running a number of different anti-spyware programs on my PC. It is a great program.

Re:It can be a bit more complicated than that

[airjrdn]

It doesn't. Syncback is a solid product, I've used it for minor backup/syncing tasks for quite some time.

For backups however, I use Acronis True Image.

Re:It can be a bit more complicated than that

[rdeadman]

I'm interested. Why not just put your work files onto a RAID drive? That's what I do on my file server. That way, things are backed up instantaneously, not just once a day. It doesn't provide off-site backup, granted, but I don't see how your set up does this either...

Re:It can be a bit more complicated than that

[thomasa]

That is not a backup. It is just a more reliable disk drive. What happens of the room your RAID drive is in burns up? What if someone deliberately writes over your RAID drives? What if someone wants to go back to a version of a file from last month?

Re:It can be a bit more complicated than that

[rdeadman]

True, but I was asking about RAID in regards to the person who does a fancy copy to backup directories on the same drives every night. That's why I noted that neither solution provides off-site backup. His solution doesn't support off-site backup or backups from last month, either. Granted, it does solve getting a yesterday's version of a file that was deleted today, but if he doesn't catch the erroneous deletion until tomorrow, he is SOL.

I'm more worried about backups from a hardware failure point-of-view than erroneous changes. In that situation RAID is better than once-a-day backups onto the same devices.

Re:It can be a bit more complicated than that

[WARM3CH]

Yes, I could as well configure the hard disks in a RAID configuration. The advantage would be instant mirroring of everything written on the disks. However, I don't really need that. I would lose lots of disk space in RAID. I have 2x160GB hard disks. In a RAID configuration, I woud have only 160GB free space (ie. 160G / 160G = 100% overhead). In my case, the total size of the "important data" is now only round 8GB. I have two copies of the backup, so I'm wasting only 2x8GB = 16GB for backup. The backup directories are actually NTFS compressed folders so the real space uesd for each one is just 5GB instead of 8GB or 10G in total. So, now I have only 10G / 320G = 3.1% of overhead! :)

Re:It can be a bit more complicated than that

[WARM3CH]

In that situation RAID is better than once-a-day backups onto the same devices.

RAID has no advantage in this regard. As I said, I make two backup copies: one on each drive. If there is a hardware failure, there is always a safe copy on the other drive.
I use this method exactly to avoid the hardware problems without the high overhead of a real RAID system. About versioning, well, I have all my codes in VSS and the whole VSS database is also part of what I mirror in backup folders.

Re:It can be a bit more complicated than that

[Jim_Maryland]

In that situation RAID is better than once-a-day backups onto the same devices.

I don't believe the poster was backing up to the same physical device (at least I hope they know better than that for hardware protection). I got the impression he had two physical disk to protect only against hardware failure.

As to the versioning issue, I'd hope the poster was using something like CVS (which is what we use internally for our development group) or any other version control application. If the poster uses a version control application, then making the backup or using RAID makes a bit more sense but does leave the "disaster at a sight" problem still. Of course the off-site backup may be overkill for what they are working on so I wouldn't hold that against them.

Re:It can be a bit more complicated than that

[fungai]

Now I use SyncBack [2brightsparks.com] which is a freeware program with all these features that I need (and more! like FTP and compression to Zip, etc.). QED.

rsync is an open source program that can do all that, and is smart about the way it syncs files. It only syncs the actual changes in the file, and doesn't copy the whole file from scratch. So if you changed 100 bytes of a 50MB document, it will only write the 100 bytes of changes. Really an amazing program written by some of the samba guys.

Re:It can be a bit more complicated than that

[Anonymous Coward]

What a freaking idiot, you don't get it do you? What happens if you make several important changes to a file and then wait till 12 for the automatic backup to occur, but in between the changes and 12, your first hard drive blows to smitherines? What happens then? RAID DOES HAVE AN ADVANTAGE, everytime you write a block to the array, all drives get written. That's the whole point, one can fail, and you don't lose your changes.

Seriously dude, come on, saying RAID has no advantage is a little stupid.

-- Chai Chai

Re:It can be a bit more complicated than that

[Jim_Maryland]

True that the RAID configuration of two disk isn't very efficient. In some cases, this is necessary, but it doesn't sound like your a candidate for a RAID really. Where you do see a better balance (cost vs reliability) is when you start looking at RAID 5 and typically lose a single parity disk out of a larger number of disk (5x160GB disk where you lose 1x160GB for parity and have roughly 4x160GB for data). Given the size of your data, making a RAID probably isn't the best solution in your case. I'd still recommend some form of version control but even that depends on the complexity of your project.

Re:It can be a bit more complicated than that

[WARM3CH]

I said RAID has no advantage is this regard. I know where to use and where not to use the RAID. You'd better learn when to use what tool for what problem. This is a system used on single workstation, not a company's webserver or whatever!
With your scenario, such disaster would at most set me back for one day worth of work. Fine, given the probabilities of it, I'm happy that my very inexpensive solution (i.e very low overhead) is quite acceptable. And guess what, on you beloved RAID, if I by mistake ruin or delete a file (which happens on a development workstation quite a lot), I'll lose it for ever, but with my method, I have at least 12 hours to recover it!

Re:It can be a bit more complicated than that

[WARM3CH]

Sure, you're right. In my a case a true RAID would be overkill. This solution of mine is really just a poor man's RAID!
And yes, I use a version control system also. The database of the version control system is part of what I mirror in the daily basis.

Re:It can be a bit more complicated than that

[Anonymous Coward]

VSS? Jesus, you're begging for it. Think there's a reason that event the VSS team doesn't use VSS?

Re:It can be a bit more complicated than that

[WARM3CH]

What's wrong with VSS? This is a single machine, and I use Visual Studio to write C++ code and MATLAB for developing DSP algorithms. Both IDEs integrate very well with VSS and the only moment I even feel the presence of VSS is when I create a project and add check a check box to add it to the source control.
I can also use CVS with the same level of integration in these tools? I don't know. Maybe yes, but why should I care if it is already working with no problem.
Sorry, but I'm not part of any cult. I don't care who has written this too. If I have it and it works and can solve my problem, then I use it.

Re:It can be a bit more complicated than that

[lgw]

If this is any good it keeps multiple snapshots of your file system taken regularly, not just one mirror. That allows you to recover user mistakes (80%+ of recovery requets come from user mistakes, not bad hardware). Otherwise, it's pretty pointles.

Re:It can be a bit more complicated than that

[orasio]

It has happened to me _twice_ before, a power failure destroying everything in the computer.
Your power source could die and take with it both your work disk, and your backup disk. Q.E.P.D. (spanish for "R.I.P.").

Re:It can be a bit more complicated than that

[Profane MuthaFucka]

Use unison, it's better, more secure, and easier to use on whole filesystems.

Re:It can be a bit more complicated than that

[WARM3CH]

Well, yes, it is possible. There are other (real) possibilities too. However, the question is as always how much you want to pay for how much security. I didn't claim this is a method to answer all possibilities. Maybe it is possible to find such a solution but surely it won't be cheap.
This method has been used on a single computer, without the possibility to access to a remote backup server. Also, size of the data is relatively small (only a few gigabytes) so an expensive fault-torlent system like a RAID5 server with a dual power supply would be an overkill. This method is usefull for users at home or small offices where even cost of a second hard disk is an issue to discuss and think of :)
I was just tring to say that a Disk-to-Disk backup system can be a little bit more complex than just a simple copy command. That's all.

Re:It can be a bit more complicated than that

[orasio]

Nowhere did I say that your method is wrong. I was just pointing out that a disk to disk backup can give you a lot of confidence that doesn't reflect the truth.

Believe me, in my country, even buying a second HD is quite difficult. I backup to CD, when I can.

A good backup disk, though, would be to have that second disk in an usb casing that is just connected for backups. The cost is not that high, and the security is much higher. Of course, that is when the data in your computer is more expensive than the 200 dollars for the HD / usb case combination.

Plus, if I were you, I wouldn't use freeware. It's hard enough to trust big corps with your data, it's harder to trust some guy's .EXE. Free software is the way to go!! You can use tar.exe + bzip2.exe

tar cv "dir1" "dir2" "dirN" | bzip2 > h:\backup.tar.bz2

That's how _I_ make my backups, at least.

Re:It can be a bit more complicated than that

[PowerKe]

Why not just put your work files onto a RAID drive?

Raid is not backup! Suppose you accidentally delete a file, RAID will delete the copy immediately on both disk. In case the files are copied to another directory you still have that copy ofcourse. But what if something happens to the partition? I've seen cases of partition corruption (fat, ntfs, ext2/3) due to power failure where you'd have to spend a lot of time trying to recover files. Using RAID you'd have corrupted both the original and copy, if you're using two separate disks with separate file systems you'll stand a much better chance.

RAID is good for hardware failures, in other cases you still need backups.

Re:It can be a bit more complicated than that

[BigBuckHunter]

When did MS remove NTBackup from windows? Why is everyone suggesting 3rd party software?

Re:It can be a bit more complicated than that

[Anonymous Coward]

It's not cult talk you moron, it's about reliability. VSS has a nasty habit of shredding old versions of files or making the current version unretrievable.

And again -- if VSS is all that and a bag of peanuts, why does no one dogfood it?

to Disk?

[rossdee]

"Please insert disk 2 of 1,270,196 in drive A: and click continue"

Re:to Disk?

[MagPulse]

I actually backed up my hard drive to 216 floppies once. I used MSBACKUP, which thankfully skipped the files that couldn't be recovered due to bad floppies and restored the rest.

No one believes me though.

Re:to Disk?

[dtfinch]

No one believes me though.

MSBACKUP actually worked? It could skip bad floppies? WTF!? No way! In my experience, the smallest of errors will make any Microsoft program blow up.

Re:to Disk?

[noidentity]

"Please insert disk 2 of 1,270,196 in drive A: and click continue"

I don't think my drive is tall enough to work with that program.

Great!

[Mad Merlin]

A beta Microsoft product for backing up all of my critical data! Where do I sign up?

To all the morons bitching about an expiring BETA

[mrRay720]

It's a BETA - use it in a production environment and you deserve whatever bad things happen to you.

Now *really* dangerous product groups with pre-programmed expiries are foods! They're not even marked as BETA! Go waste your time bitching about those non-BETA products that expire even though you've paid for them instead.

Re:Expiring Backup Software?

[R.Caley]

So, they will be installing an "easy backup" application, potentially in a mission critical environment, that will expire at a predefined date.

Best that these people get to make this kind of decision as soon as possible so they can be sacked and can go on to a career they are more suited to, for instance burger flipping.

Installing beta test backup software on production systems?

Linux has had this for years

[Anonymous Coward]

Your example is funny, but there is a valid point there.

My son does contract PC support work, and, for the last couple of years, he has been using Linux to make disk-image backups of his customers' Windows PCs.

I'm surprised it took Microsoft so long to provide similar functionality.

RAID

[mstandfest]

couldn't you just do that with RAID and not have to pay even more money to microsoft? i understand that isn't the point of RAID, but you can use it to backup your data on the hardware level, instead of whatever buggy software they come out with next.

Re:RAID

[PinkX]

And how is your RAID going to help you when your data loss is due to an (un)intentionall deletion, and not some kind of hardware failure?

Re:RAID

[zaphod123]

If the RAID box happens to be a Network Appliance file server, it would work... :)

My previous employer had NetApps. In three years we only recovered a file from tape once. All of the other times it was from a snapshot.

Re:RAID

[mstandfest]

if you have two drives that are mirrored. you could swap one of them out as often as you wanted a backup. so as long as you don't throw out the hard drive you have a backup. that is how it will prevent loss due to an (un)intentionall deletion.

Subversion as a backup solution

[damieng]

I've recently been using Subversion as a backup solution at home with great success.

My server runs it's own SVN repository and each of my machines can check in it's important files into the tree.

This backup solution is quick and thanks to tools like TortoiseSVN integrates into the desktop for ease of use.

Additional bonus factors are the ability to see the revision history, roll-back, full cross-platform support.

You can also manage multiple copies of the same file to multiple machines should you need to work on them or just want additional resilience.

The real icing of the cake of course is that you can run it over SSL via Apache or over SSH and therefore remotely access your backed-up files from out on the Internet should you suddenly need an invoice or a photograph while sitting in a net cafe in a foreign country.

Oh, and it's free by both definitions. http://subversion.tigris.org

Re:Subversion as a backup solution

[Anonymous Coward]

Will it back up locked files? Hmm?
Will it copy the entire filesystem so you can just drop in a replacement when your primary drive fails? Hmm?

Re:Subversion as a backup solution

[Anonymous Coward]

If you do this with binary files, your repository will grow quit large very quickly.

Watch out, there is a exploit in the wild when using apache and svn.

Re:Subversion as a backup solution

[damieng]

No, it's a source control system for files, not a disk imaging system.

Re:Subversion as a backup solution

[damieng]

The binary files I use don't tend to change very often with the exception of Office documents.

I'll check out that exploit before I open up my svn to the outside world ;-)

Re:Subversion as a backup solution

[malloc]

AC wrote:
If you do this with binary files, your repository will grow quit large very quickly.

damieng wrote:
The binary files I use don't tend to change very often with the exception of Office documents.

Of course it should be made clear that Subversion uses xdelta to get effecient binary diffs. Your repo will only grow quickly if the binary files are completely changing (e.g. compressed so all bytes change). If the file just gets something tacked on the end then you'll only be storing smaller diffs.

Malloc

Re:Subversion as a backup solution

[Profane MuthaFucka]

Unison is better for the same purpose. It also works over SSH.

I can see the transcript window now.

[zwilliams07]

08:40:33AM - Connecting to Microsoft Mothership.
08:40:45AM - Connection to Mothership established.
08:40:45AM - Server Message: "Welcome to Microsoft."
08:42:02AM - Uploading source data to target.
12:55:54PM - Logging and Catagorizing.
12:55:55PM - Invading your privacy.
12:56:01PM - Stealing ideas and using them against you.
12:58:34PM - Patenting all data as M$'s.
01:13:32PM - Server Message: "Suckers."
01:13:32PM - Connection closed at server's request.

Off topic?!

[Ulric]

The article is about a backup solution for Windows servers. The GP writes about a backup solution for Linux+Samba, which is often used as a replacement for Windows. How is that "totally" off topic?

Re:Expiring Backup Software?

[smkndrkn]

We're also aware that system administrators often commit horrible -- albeit accidental -- decisions in a pinch.

Only poor ones. As as systems administrator I would never use a beta like this to backup ANY data that was important.

You really have bigger problems with budget/manager/etc if you don't have a better backup product to use.

Re:I understand.

[Anonymous Coward]

Down here in Miami we have this thing called "Cuban Coffee" it comes in a small cup about 1/2 the size of a shot glass. It's a kick in the pants. You, my friend need some.

Tell me how big your tape backup is. I've got a 3 terabyte server with a database on it that's in the 60gb range, and just over a terabyte worth of other data. Now, again, tell me how do I fit that onto a single 200gb LTO tape??

Let me know when you've had that coffee and we'll talk again!

PS: My guess is that Microsoft's solution still isn't fast in this case, and guess what -- I *still* need my tape backup for disaster situations.

Re:how does it compare to Norton Ghost

[jasonaedwards]

it costs 50 dollars, isn't beta, and it works :)

Seriously, Ghost 9 is great, and creates images without the need to reboot your PC/Server. Picked up that ability from Drive Image when Symantec bought out Powerquest.

I don't get it.

[JayJay.br]

If you really need the uptime, you may already have a storage unit, which is almost certainly capable of snapshots/snapclones with close to zero downtime (some of them don't even bother copying the full contents of the drives -- just the differences!).

Anyway, this would be only for databases, AFAICT. Any other kind of data usually does not need that kind of bringing-down-the-server-for-backups consistency.

So, what's the point? Is this to be sold to enterprises that are so small that don't use storage systems, and most certainly don't need the uptime?

No, really, it doesn't look like MS will get much revenue from this.

RTFA

[AnomalousTurd]

It seems like a decent feature set. A sort of scheduled mirroring of volumes over a LAN, utilising the Shadow Volume copying. An agent runs on the source server and logs the data changes. At scheduled times this agent transfers the accumulated changes to the DPM server.

The server can produce snapshots etc and there seems to be some tie in to standard file save/open dialogs so users can access previous versions.

Disk manufacturers will love it :-)

How much to bet? ....

[Stephen Samuel]

That they've managed to get a patent for this 'idea'?

Re:How much to bet? ....

[davidsyes]

They probably did, and they'll probably try to claim prior art from way back to windoze 3.1 days...if not stopping at win98 se

Considering the past and represent behavior of this past recidivist, recalcitrant, intransigent, convicted monopolistist, why was Samuel's comment/remark marked as "flamebait"?

After all, this is microsoft/mshaft (say it in the tone of "THESE ARE ROMULANS! You show weakness, and they'll come back with not just ONE ship, but with EVERYTHING they've got! You're the expert modder who marked him flamebait; you KNOW this; why are you chopping him down?) (lower-casing/deprecation of ms' name intentional/perpetual with me...) we're talking about. Doesn't MATTER that ms "donates" (read: we'll donate you computersif you agree to never change the OS, just like we bend manufacturers over the barrel (even tho some are still trying to get to Linux after they've mostly dumped Solaris...) schloads of money or s/w with pre-arranged hardware drops. Once you've destroyed companies (bankrupted via vaporware, sued based on nebulous claims, lied to judge with fake video testimony in court, affecting possible outcomes of companies trying to make 3rd-party add-ons if the verdict lands in favor of 3rd-pty s/ware makers...)

David Syes

Too late

[dtfinch]

I've already switched to samba and rsync. Microsoft's backup was outdated by at least a decade, and even failed to complete at random when I've used it for disk to disk backups. And Windows' mandatory file locking policy makes safe, reliable backups entirely impossible. An xcopy backup is even dangerous, because it temporarily locks files as it opens them for reading, potentially causing other server processes to fail if they attempt to write to the files.

backup strategy

[Anonymous Coward]

we haven't looked back since we've moved to vmware esx server. it makes life so much easier. we can take snapshots of running vms with esxRanger with ease. the virtualized environment and abilities provided therein have really boosted our DR/HA capabilities. if you haven't used esx server i recommend you try it.

Re:I understand.

[jmcmunn]

Here's a link to a similar tape drive. Ours is slightly older, and a bit smaller I believe, but I'm not the guy in charge of nightly backups, so I can't give you the exact model at the moment...

http://www.superwarehouse.com/Exabyte_Magnum_6x60_ _LTO_Tape_Library/270032-100/p/404584

An image in a state of flux

[Pac]

It would be a poetic mental picture if we're not talking about fat, overworked, graveyard-shift sysadmins trying to finish their nighties and go home...

Re:An image in a state of flux

[operagost]

nighties

I hope you meant "nightlies." You just gave me a very disturbing mental image there.

Hell no

[dtfinch]

From the FAQ:
a customer has to purchase a server license for every DPM server that is deployed and a Data Protection Management License (DPML) for every server they protect.

Now they have incentive to never upgrade the poor quality backup software already included in Windows. Admins will have to buy their backup software seperately or look elsewhere. Server operating systems are expected to come with _good_ backup software, so from a strictly technical sense Microsoft is being an ass.

"easier than simply backing up to tape"?

[mordejai]

It looks like you are backing up data...

Rsync works GREAT for me

[VON-MAN]

Yep, i'm doing the same on my desktop at home. A snapshot of my /home/VON-MAN to an extra harddisk every 4 hours. The great big advantage of doing it this way is: i can turn my computer of when i'm done with it. I don't have to worry about a backup scheme, it's "set and forget". I don't have any problems with tapes (don't like 'm). I have snapshots of my data up to a month old (good enough for me). Older stuff is burned to dvd. And restoring is as easy as cd-ing to the right dir and copying. One snapshot takes about 2 minutes (i'm not really sure, i hardly notice it) of disk activity to make.
I exclude dvd and movie files and iso's, this makes one snapshot 17G (my home is 34G), and all the snapshots combined (15) are 25G.
So, for the price of a small harddisk and some cycles i've got the perfect backup. I have been burned by harddisk failures two times (and lost some stuff) so i'm pretty anal nowadays.

Start here with rsync backups:
http://www.mikerubel.org/computers/rsync_snapshots
Jaap

You're missing something

[extra88]

It's not doing a sector by sector copy from one disk to an identically sized disk, it's copying the files from one drive to space *within* another drive. The receiving drive (or "drive") will most likely be on a separate server with space to hold multiple full and/or incremental copies. This is a very common feature of backup software, they do backups to more than just tape.

What makes this newsworthy is it's Microsoft so they'l likely undercut the prices of their competitors and use their insider knowledge of Windows and their server software to ensure that data (especially things like MS SQL server) is backed up in a restorable state with minimal/no downtime. Also, I think if it's integrated with the Volume Shadow Copy it can improve that technology's ability to allow restores by users (vs. admins) by offloading the shadow copy from the server to a dedicated backup server.

Linux comes with backup software?

[Anonymous Coward]

Hi!

You all seem to bash MS again... .. but tell me, where can I find an useable Backup program from my SuSE 9.2 Professional? Windows 2000 Professional as well as Windows XP Professional both have a good schedulable backup program (included free as it should). But there is nothing on SuSE. (Ok, there is tar, but that definitely does not count! And then there is that on system backup in the YaST, but even that doesn't come close to what a backup program should be like - in order to be useable.) So, in terms of backup software, MS seems to be way ahead of SuSE, which is about the best regarded distro nowadays...

Re:Linux comes with backup software?

[Anonymous Coward]

Yeah, their idea of scheduled disk to disk backup would be "dd + cron".

Re:Linux comes with backup software?

[Anonymous Coward]

What about non disk to disk (I know that's what the article is about, but...)? There is no backup software, right? At least not free as in windows?

Re:Linux comes with backup software?

[repvik]

If I'm not mistaken, SuSE comes with LVM-support (Atleast 9.1 did). That's about all you need for a snapshot backup, without needing to shut down anything. What you do with the snapshot volume is ofcourse your choice... You can "dd" it somewhere, you can use tar, afio, cpio, rm -rf /... whatever beats your meat.

There's no "click-n-drool" interface backup, no. Maybe it's because there are so incredibly many ways to back up the system.. I dunno. But it wouldn't take me long to set up a proper backup system in SuSE. All it takes is a google search or two (Google is *still* your friend! (Or Yagohoogle)).

The point of GNU/Linux hasn't been user-friendlyness. It's flexibility, power and reliability. Linux still sucks on the desktop, and will continue to do so for atleast five years.

Microsoft now provides a solution that possibly gets close to the power and flexibility of unix backup systems. But it probably has a UI that makes it a lot easier to set up.

Both sides have their advantages. Setting up a server is quicker with windows. Setting up a *good* server is easier with *nix. Same will probably apply for backup systems.

Gah... I gotta stop raving on about this. Sorry for the inconvenience people. I'm stopping now ;)

Re:Linux comes with backup software?

[Anonymous Coward]

The "Free as in Windows" software you refer to (which, by the way, is not free) does little more than dd with cron. `dd` is for straight disk reading, but where it goes is really up to you. Tar to tape; mkisofs for DVD; whatever. And beyond this one could use rsync for incremental backups uploaded to an offshore located (encrypt/compress before sending over the line).

So do the GNU/Linux [still free] products beat the included Windows Backup Software? I'd say yes. I've used both, and the MS backup utility is extremely basic.

Re:Linux comes with backup software?

[Tsiangkun]

It really depends on what the word "backup" means to you.

You might be able to use rsync to make your backups with very little effort.

Re:Linux comes with backup software?

[Anonymous Coward]

The Win2000/XP backup program is quite far from basic. With that I can do scheduled backups from selected folders. I can also restore them to original localtion or other locations. And I can do all that (and more) WITHOUT GOOGLE or MAN! Just by looking at the program. NOW where is linux program that I can do that.
And yes, it is free. I paid for windows and I got that. I also paid for SuSE, and I got nothing.

Re:how does it compare to Norton Ghost

[hey]

Doesn't Ghost let you swap out OSes?
I bet the Microsoft tool doesn't do that!!!

Seems like a ripoff

[Guspaz]

Seems like a ripoff of this Linux-based solution:

http://nitix.com/technologies/technologies_idb.p hp

Re:how does it compare to Norton Ghost

[gbjbaanb]

Actually, Ghost 9 is fantastic, it now creates incremental backups.. which on my home PC, completes in less than a minute. (ie. I don't really notice when it kicks in).

Add excellent scheduling and I really recommend Ghost 9.

g4u

[Anonymous Coward]

I can't believe nobody has mentioned g4u.

http://www.feyrer.de/g4u/

Full disk backups, supports FTP, compression, dual boot, and any x86 operating system.

Re:A few points - Don't need no downtime

[Anonymous Coward]

You don't need downtime to do a decent backup - if you've got the right gear.

At the big sites I've worked at, we used to have disk mirroring for backups.
We would sync the mirror disks, take them offline, & back up to tape, then put them back online to resync with the main disks.

This lets you do full backups & database checkpoints while the main system is still running. The main system's on raid, so you're still covered for disk failure even when the mirror is offline.

Beta?

[TheSpeedoBeast]

I don't think my heart could handle using a beta for backup software

Nothing new in the world...

[maird]

...at least from Microsoft perhaps. This sounds like a straight copy of Novell's data protection server feature and, no doubt, a few other technologies from various platforms. I wonder if they'll even copy the Explorer shell integration? I think the Novell one is a bit more than RSync and traditional backup but is limited to Novell's file system (NSS) as I understand it. It's more like CVS meets backup. It is disk to disk but the target disk isn't a duplicate of the source (i.e. not COPY, XCOPY or RSync). The target is a database of versions. I think Microsoft's is similar. In NSS the file system internally tracks changes and can give you them in a list (doesn't need to be trawled for changes). It supports snapshot with copy on write at the file and volume level so you can exclusively lock a file without preventing it from being changed (though you have to have some knowledge of concurrency among your applications to use it safely). Files retain all security attributes in the archive. Users can do their own restores from their workstations (Windows ones get a context menu option) and their access rights are enforced in the retore software so they only see files for restore that they had righs to originally. The common restore case: single file due to user error I believe. If users can restore their own files you don't need to recover a tape, cue it and wait for it to spool to the file in question.

NTBACKUP is JUST FINE

[mrsbrisby]

We use NTBACKUP and mtftar with great success. With a SYSPREPP'd install disk, restoring a kaput Windows machine takes about 20-30 minutes. With mtftar, we can cherry-pick files without booting a windows machine.

NTBACKUP can create ".bkf" files that surely, aren't as fast to create as shadow-volumes, and incrementals aren't useful for large files (like rsync), but they have advantage is that they can be stored on a remote machine on the cheap!

Be afraid, be VERY afraid

[XO]

Considering that Microsoft's releases of software tend to be somewhere around most other company's late alpha or early beta testing cycles, quality wise... I don't think I'd want to even think about trying a Microsoft beta data backup product! I can just see it corrupting the backup and the original, simultaneously!

Rsync Gather-Scatter backups over ssh and local

[the_olo]

I'm using the following method:

I have a directory tree where I keep all the important data.

In its root there is a Makefile and a set of scripts that are called by that Makefile.

When I go to that root directory and run "make", what happens next, is:

A list of data repositories available from the current machine is read (cat repos.$(hostname)). I have different sets of reachable repositories at work and at home...

Then, in a loop, data is "rsync -avu"-ed from all repositories. They may be remote ("-e ssh" gets appended), they may be local.

I receive only the new/modified files from those repositories.

Then, in a similar loop, data is "rsync -avu"-ed TO all repositories, so that new/updated data files collected from repositories, get distributes in the identical form to all the other repositories.

I refer to this algorithm as "gather-scatter".

This way, I have 4 mirrors of my important documents at work, and 3 mirrors at home. They are kept in sync.

(BTW, since I don't use the "--delete" option, It gets hard to delete an important file from this distributed storage. Either accidentally, or on purpose ;))

Never lost important data since when I've implemented this.

Re:I understand.

[lgw]

Sony's SAIT is 500GB native (1TB compressed is a pretty good real-world estimate). If you use actual backup software, of course, you don't worry about fitting everything on one tape (or worry about what tape has what data, etc).

For larger backup requirements, just get a tape library and good backup software - this has always been the best solution, if your data's worth over 100K.

Re:how does it compare to Norton Ghost

[Anonymous Coward]

Actually, a critical portion of the technology behind Ghost 9 (the snapshot driver) appears to comes from a company called Storagecraft. If you check the file properties of the pqv2i.sys driver that's included in Powerquest's older Drive Image and V2i Protector products, and which is also in Symantec's Ghost 9, you'll find that it's copyrighted by Storagecraft.

Ghost 9 is a good product, but not much credit can be given to Symantec for this. Most of Symantec's technology was really developed by other companies, such as Storagecraft and Powerquest in the case of Ghost 9. Kudos to them.

Already have this

[zenst]

Its called Raid mirror's using hotswappable discs.

1) take disc out

2) insert spare disc

3) disc that is taken out is verbatim backup of what you have, so go take offsite

4) Spare gets mirrored with live data by the raid to resync it.

5) cookies allround.

Coz the real secret is to be able to pull data back and in an organised way, this is were all great backup plans usualy fail.

Now had they said they were going into the heriacial storage game with nice cheap solution and nice end-abUser interface then I'd be going yeah great, go Microsoft. But it aint and I shant.

Until then...

GNU tar with MS User Interface?

[rice_burners_suck]

Hmmm... This must be GNU tar with a talking-paperclip interface... or actually, maybe they'll use a talking hard drive. And the program, I'd bet, is requires 500 megs of hard disk space for compact installation, and a gig of RAM to execute.

Yup. Sounds like Microsoft. Where do you want to go today?

RAID1

[gnuman99]

I have a disk-to-disk backup too. I call it software RAID1 - no downtime!!

As for code, I do one better, I use a version control system so I can replicate changes on any number of computers at commit.

i fail to see the news...

[sad_]

...in this one. other backup software has been doing this already for years. i have been doing it for a year now. the article mentions CA and Veritas as two of the companies with backup software which is capable.
so, this is news because; microsoft released something?

What the hell happened to...

[Anonymous Coward]

RAID 1??

I'm no tech wonderboy but what the hell happened to RAID 1??

Or hell, RAID 10 (RAID 1 with redundent controlers?). To hell with all that even... why not go with a script that just dumps EVERYTING over the net to a backup distributed server?

All this mess of backing up. Sheesh. Gmail GIVES you 2 gigs of free space. Why not link a few hundred gmail accounts together and a script to link them all together useing GmailFS to make it store it? (Note: is all a functional wonderment, not a production based idea.)

maxwell, meet bzip2

[Anonymous Coward]

duh.

Professional backup systems

[Anonymous Coward]

I haven't read all of the submissions in this thread, but here are some general comments from someone who works on a team who design and manage 1PB of online storage (large UK bank, can't name it etc. etc)...

1 - RAID is NOT backup. Raid is disk protection, a backup is something which will enable you to recover files from a given date (within retention) not make hardware continue to operate even though there has been a failure.

2 - Disk to Disk backup, or disk staging, means that the data moved by the backup software is initially stored on a disk attached to the backup server - it is not a fancy copy command. More offen than not, the data is moved from the disk to tape storage at a later time.

3 - Disk imaging is all very well for someone at home, but in Enterprise environments you can't down a system and copy the entire image of the attached data somewhere else. Also you don't get differential or synchronous backups so it's the full image each time, rather than deltas, this quickly gets unmanageable.

4 - You DO need to down various applications in order to back them up. The vast majority of databases need to be either downed or have a specialist agent installed to move the data to the backup system.

5 - Nobody in a half tech-savvy business backs up to DVDr, CDr, usb attached devices or the such like. Big business backup systems generally utilise several powerfull servers which move data from multiple clients to disk and then tape systems. The tape systems in question are usually robotically hosted and these days offen attached to a tape SAN which allows sharing of the tape drives between hosts.

6 - MS's product seems to work in a similar way to TSM one of the 'big three' backup products (Veritas NetBackup, Legato Networker and IBM Tivoli Storage Manager) Of these three, TSM is unique in that it only does one full backup - the subsequent backups are all deltas, this is transfered to disk and then to tape. There is various jiggery-pokery (TM) which copies data from differently aged tapes in order to keep the minimum amount of tapes with data stored upon them.

err... That's it. Rant Over.

PS You really do have to down lots of systems in order to back them up effectively. No matter which OS is running them.

PPS People making comments like 'maybe MS should make a system which is reliable enough not to need constant backup' should a: Get a life and b: see the cost of recovering their LINUX/UNIX/MAC OS/Windows etc. hard disk when it crashes and looses their data.

PPPS Everyone should backup!!! Really, please, backup, apart from anything else, it keeps me in work!

Re:I understand.

[Anonymous Coward]

Usually, in the real world, we work on 1:1.5 compression in a tape drive. Sony's drive is impressive for capacity, but it does rely upon helical scan technology, rather than the more robust linear scan (DLT, LTO etc)

Tape libraries are not that expensive these days, you can get a StorageTek L20 for about three or four grand (Sterling) add to that the cost of one or two drives and yes, it does leap a bit, but you've already got one drive so you can partex that.

Personally at my work, we have four L5500 silos with 30 LTO-2 drives in each. An L5500 costs somewhere in the region of a million quid, but they are cool - there is a camera on the robot arm, which displays on a monitor!

On-the-fly backups (dump -L on FreeBSD)

[ge]

I've been using live backups on FreeBSD for a while now, and I have had 0 issues with it. The way it works:

- dump requests snapshot of the volume being backed up
- file system creates *consistent* snapshot of file system (volume managers can't do this!)
- dump makes backup of snapshot
- dump removes snapshot

Just pass the '-L' flag to 'dump'.

If you use another backup utility you may have to write a script to create a snapshot, and mount it through a loopback device. Nothing a good sysadmin can't handle.

It works great. Just make sure your datebase transaction logs are on the same volume as the main database :-). After a file system restore the transaction mechanism of your database should get things consistent again.

Re:how does it compare to Norton Ghost

[Anonymous Coward]

Yes Definately, They use it at my School becasue all they have to do to change a class from a server 2003 class to an A+ or whatever is broadcast. In like 10 minutes complete new OS.

Re:how does it compare to Norton Ghost

[planckscale]

From what I remember with Ghost, I need to reboot in order to boot the machine to a floppy and dump the image to my Ghostcast server. How do you create the image without rebooting?

Re:how does it compare to Norton Ghost

[Danathar]

Why am I marked troll? I was'nt implying one was better.....just asking about the differences

why make it complicated

[Anonymous Coward]

backing up data should be obvious to any internal details, which are already handled by the file system, thus being only responsible for the external detail which is a simple byte stream, recording it and playing it back.

Donald Ray Moore Jr. (MindRape)
damageD Cybernetics

Move to Disk-to-Disk Continues

[RonBurk]

Microsoft has long had a toe in the water with some sort of modest backup .exe included with Windows. But their previous announcement of a full-fledged disk-to-disk backup solution meant that they now see a lot of money flowing into the backup software market, enough to make them want a big piece for themselves.

You can also view this as the typical Microsoft "why don't you partner with us since we don't want to do that -- oh my, look at how much money you're making -- sorry, we have to crush you know". In this case, the hapless partner is Veritas. Last time I talked to someone there, the slightly nervous attitude on Microsoft's threatened rollout was essentially a hope that customers will not really want to trust their data to a Microsoft product.

They may be right. It's certainly somewhat unusual for someone to roll out a big public beta of a backup product -- people generally want to get the feeling that their backup software is free of data-losing bugs at v1.0, not have to wonder if all the 567 bugs reported in the beta period really got fixed and fixed correctly.

If the Microsoft DPM product actually ships, I'm sure Veritas will have some sleepness nights waiting to see if it takes some of their market share. And probably a few more sleepness nights waiting to see if Longhorn will provide some undocumented hooks to Microsoft's product that gives it an edge in doing things like hot backups of particular kinds of data Windows uses.

There seems to be some confusion about hot backup in this thread. Basically, you cannot do a hot backup of an entire disk that maintains integrity without the cooperation of any software that might be running. There's just no way backup software can magically intuit that third-party software is in the midst of a logically atomic operation that required multiple writes to the disk.

Database software that uses transaction protection (which is a minority of all databases on the planet) can, of course, recover integrity even in the face of a backup that does not coordinate operations with it. But the kind of people who would buy Microsoft DPM would probably be running a database that comes with its own hot backup solution.

But the big news here is simply that disk-to-disk backup cannot get more mainstream at this point. It's rapidly gone from a kind of novelty alternative to tape in certain situations to hot new product to so-boring-that-now-even-Microsoft-does-it. There's no shortage of tape installations that will persist because they are simply good enough, but it's getting tougher to find enterprises that are planning a big investment in a new tape backup system.

Re:how does it compare to Norton Ghost

[Anonvmous Coward]

I caught it in meta-mod and marked it unfair.

Wish I could do more.

Re:Expiring Backup Software?

[Anonymous Coward]

Its a beta product...

Now, unlike open source software, where version 0.0001 is the final and full release people use, MS does at least try things out and often lets other people have a go with them in the wild before releasing the product as fully tested.