Complete Filesystem Checkpointing?

polymath69 asks: "Living on the edge of Debian unstable means that updates sometimes break stuff, occasionally to an extent that is difficult to recover from. This got me thinking about treating the entire set of mounted filesystems as a transactional database. Mark state, try something which might be dangerous, test, and approve (commit) or panic (rollback). Obviously some filesystem support would be required, but with ext3 and reiserfs available, maybe the potential is already there. And such a system would need lots of disk space, but these days that's a demand easily granted. There's lots out there on process-level checkpointing, and even some stuff about system-level checkpointing, but all I've found on that was in the context of saving and restoring processes for a system freeze and restore. But I couldn't find anything on Google or SourceForge about doing this sort of temporary branching in the filesystem. Is this idea feasible? Is anyone working on it?"

Comments

Chuq is working on it.

[omega9]

Well, looks like this guy Chuq is working on it. He seems to be a kernal hacker that works for VERITAS.

You can also find interesting filesystem info here

There's also work being done on TRAM (Transactional RAM).

Seems simplistic but...

[omega9]

It's a bit time consuming, but tar has been doing this job for me for a while. I finally (took me a long time, ok) created a working bootable linux CD that mounts my filesystems RO and tarballs the entire contents to a remote server. It takes about 1h20m to do a full backup.

After it finishes, I test software on the existing system. If it breaks I restore, if it doesn't I play-test it for a while and if it keeps behaving I commit it to the next full "state preservation".

The biggest drawback, of course, is that this sceme requires close to 1.5 hours of downtime for the backup and more if you need to restore. But for noncritical systems it works great.

AFS

[William Aoki]

AFS will do something like that, although not to the extent that I hear NetApp Filers can. Off the top of my head, there are two ways to do this with AFS. Both these methods require superuser access to your AFS cell, unless backups or replication releases are being done automatically.

(CodaFS should be able to do this too. I haven't played with CodaFS enough to know if it offers any other way to accomplish checkpointing.)

Method 1: backup volumes

$ cd /afs/mycell/some/path
$ kinit me/admin
Password for me/admin@MYCELL:
$ aklog
$ vos backup some.path.avol
$ kinit me
Password for me@MYCELL:
$ aklog
$ cd avol

do stuff with the filesystem...
Oops! I need files that I modified or deleted!

$ cd ..
$ fs mkm avol.backup some.path.avol.backup
$ cp avol.backup/little-lost-file avol/
$ fs rmm avol.backup

Many sites run 'vos backupsys' (generally before 'vos dump'ing volumes) every night to automatically back up all their volumes, and leave users' backup home volumes mounted under their home volumes, to provide easy access to yesterday's files without an administrator's help.

Method 2: for replicated volumes

$ cd /afs/.mycell/some/volume

do stuff - uh-oh, I need a file back that I changed!

$ cp /afs/mycell/some/volume/my/file my/file

ok, finished with the changes. Commit them!

$ kinit me/admin
Password for me/admin@MYCELL:
$ aklog
$ vos release some.volume
Released volume some.volume successfully
$ kinit me
Password for me@MYCELL:
$ aklog

Volume (for volume, read filesystem) backups work by saving the state of a volume at the time the backup command was issued. When changes are made to the volume, the original state is copied to the backup volume. The backup volume only takes as much space as the changes made since the last backup. Replication works by making read-only copies of a volume in one or more locations, as specified by 'vos addsite' commands. The copies are only updated when changes are 'released' from the read-write copy to the read-only copies. By convention, cell root volumes are mounted read-only on /afs/cellname and read-write on /afs/.cellname.

I think that newer versions of Solaris will do checkpointing on UFS. I haven't adminned Solaris since 2.3 (the slooow SS20 with 2.8 under my bed dosen't count until I play with it some more), so I'm not familiar with the details.