Slashdot Mirror
Ask Slashdot: Asynchronous RAID-1 Free Software Backup For Laptops?
First time accepted submitter ormembar writes "I have a laptop with a 1 TB hard disk. I use rsync to perform my backups (hopefully quite regularly) on an external 1 TB hard disk. But, with such a large hard disk, it takes quite some time to perform backups because rsync scans the whole disk for updates (15 minutes in average). Does it exist somewhere a kind of asynchronous RAID-1 free software that would record in a journal all the changes that I perform on the disk and replay this journal later, when I plug my external hard disk on the laptop? I guess that it would be faster than usual backup solutions (rsync, unison, you name it) that scan the whole partitions every time. Do you feel the same annoyance when backing up laptops?"
Just buy a mac :-)
Use mdadm -C -b internal to create a bitmap. Detach and readd the mirror at will and it will only sync the difference.
RAID is not backup.
How is traversing the whole directory tree with find different from what rsync does?
Running a daemon that lists modified files using inotify might work.
thegodmovie.com - watch it
You could try DRBD (whole disk level) or ZFS with detached mirror or snapshots. Both will keep track of changes and resync only things that changes from the last sync.
OK. But you still scan the whole disk with a find command.
But what about Dr. Feelgood?
Cleanest implementation of this I've seen is with ZFS.
You do a snapshot of your filesystem, and then do a zfs send to your remote backup server, which then replicates that snapshot by replaying the differences. If you are experiencing poor speed due to read/write buffering issues, pipe through mbuffer.
The only issue is that it requires that you have your OS on top of ZFS.
You want two ZFS filesystems. One local laptop pool, one backup pool (and it really should have two disks, but one will work fine). Snapshot your laptop filesystem periodically (cron or something), and then zfs send/receive that snapshot to the backup pool when you have access.
Are you backing up EVERYTHING on the laptop -- OS and data included? Even if you are only backing up your home directory there is stuff you don't need to backup like the .thumbnails directory which can be quite large. Try using rysnc's exclude option to restrict the backup to only what you care about.
DNA
AKA mrascii
TimeMachine takes about 15 minutes to do the prep work before it starts copying for me, on a 2012 Retina MBP with 16Gb of RAM and only 256GB of disk space ... 64 GB taken by an unbacked up BootCamp part and another 120 or so eaten in Windows VMs that don't get backed up either ... i.e. Its not a slow spinning platter backing up a terabyte of data.
I see no indication of any Journal, it certainly isn't making it faster. Pretty freaking slow actually.
Persistent Volume manager for Kubernetes - https://github.com/dwimsey/openshift-pvmanager
In this case, it sounds like you want a fast on-demand sync rather than a RAID.
However, you could possibly use dm-raid for this if you're a linux user.
Have the internal disk(s) as a degraded md-raid1 partition. When you connect the backup disk, have it become part of the RAID and the disks should sync up. That said, it likely won't be any faster than rsync, quite possibly slower as it'll have to go over the entire volume.
Alternate solutions: /home is a combination of /mnt/home-ro (ro) and /mnt/home-rw (rw, COW filesystem). When external media is connected, /mnt/home-rw is synced to external media, then back over /mnt/home-ro
* Have a local folder that does daily syncs/backups. Move those to the external storage when it's connected.
CAVEATS: Takes space until the external disk is available
* Use a differential filesystem, or maybe something like a COW (copy-on-write) filesystem. Have the COW system sync over to the backup disk (when connected) and then merge it into the main filesystem tree after sync
For example,
That will still take ages...
Why not give Bittorrent Sync a go? It's a decentralized "dropbox" on steriods!
http://labs.bittorrent.com/experiments/sync.html
:. Ultimate Control Dedicated/VM Servers
Is to not try to keep 1TB of crap on a laptop... or anywhere for that matter. Travel light says me ;)
Considering how slow it is I doubt it.
I sometimes use a Mac, I still prefer rsnapshot over some backup that is likely hard to deal with if you don't have another mac.
The OP doesn't mention which OS he's on - the tools he mentions both run across multiple OS's. Would be helpful to know. I know as a group we probably assume some form of Linux but..... I use MS Home Server at the house to back up my family's multiple Windows machines. Runs on crappy hardware, does incrementals on a schedule, allows file level or bare metal restore, keeps daily/weekly/fulls as long as I ask it to. I know we aren't a Windows friendly crowd but this product does exactly what it promises and does it pretty well.
"Would you, could you, with a goat?" Dr Seuss
This doesn't match my experience. Time Machine fires up in the background, does its thing, and then stops shortly thereafter. Certainly much less than 15 minutes. More like five or less. This is on a new-ish iMac with a 3TB internal drive.
It wouldn't even be noticeable were it not for the fact that I can hear the TM destination drive (sitting on a shelf behind me) spin up once an hour.
People who say "sheeple" have about as much sophistication as an AOL user, and in fact are probably actually AOL users.
Sorry, internal drive is 2TB. Time Machine destination is 3TB.
People who say "sheeple" have about as much sophistication as an AOL user, and in fact are probably actually AOL users.
Well, all my research work from the last 20 years, that makes some data, and you never know which data you need when you travel. So, when I change laptop, I copy all my data from the old disk to the new disk. That's why also I want to backup only the diff, and not spend my time scanning the disk to find these differences (which can be spread all over the disk).
CrashPlan is free, but not open, and I think will do everything you need. You can backupto an external disk, over the network to one of your own machines, or back up to a freind who also runs it. Great key based encryption support. If you want, you can pay them for offsite backups (which is a great deal as well, in my opinion). It's cross-platform, and easy to use. Never underestimate the benefits of off-site backups.
It's different in that you don't have to sit and wait for it and doing the backup will consist of only the actual copying
I suggest you look again at rsync. /proc and there are some directories such as /tmp and /run which you may not care about).
- It compares changed files and copies only what has been changed. Changed files are identified by differing mtimes (by default).
- rsync can also handle removed files with the --delete option.
- It can do the entire filesystem tree in a single command
- There are filter options so you can include/exclude what paths to copy (eg you don't want to copy
Windows Backup (since Vista) use Volume Shadow Copy (VSS) to do block level reverse incremental backup. I.e. it uses the journaling file system to track changed Blocks and only copies over the changed Blocks.
Not only that, it also backs up to a virtual harddisk file (VHD) which you can attach (Mount) as a seperately. This file system will hold the complete history, i.e. you can use the "previous versions" feature to go back to a specific backup of a directory or file.
Reading slashdot one-liner: (irm http://rss.slashdot.org/Slashdot/slashdot).rdf.item | fl title,desc*
and perhaps moved into an archive for later time travel, making it useful as an actual backup
A better way to handle that is to use a copy-on-write filesystem and taking snapshots after each backup. That way you get the tree in each snapshot as it was at the time, and without duplicating space.
This is my current experience with mine too. However during the prep stage it is making room on my time machine drive to receive the changes. Consolidating the older files will take time.
When my drive was new and had plenty of space, the prep stage was much shorter.
These comments are my own and do not necessarily reflect the views or opinions of my employer or colleagues...
Holy cow people, your missing the OP point. It's taking 15 minutes to SCAN the 1TB drive.
I've run into the same problem on windows and Linux. Especially for remote rsync updates on Linux on slow wireless connections. It's not the 1TB that kills since I can read 4TB drives with hundreds of movies in seconds. It's the amount of files that kill performance.
My solution on windows is to take some of the directories with 10,000 files and put them into an archive (think clipart directories). Zip, Truecrypt, tar, whatever. This speeds up reading the sub-directories immensely. Obviously, this only works for directories that are not accessed frequently. Also, FAT32 is much faster on 3000+ files in a directory than NTFS is. Most of my truecrypt volumes with LOTS of files are using FAT32 just because of the directory reading speed.
On Linux systems, I just run rsync on SUB-directories. I run the frequently accessed ones more often and the less-accessed directories less often. Simple, No. My rsyncs are all across the wire, so I need the speed. Plus some users are on cell-phone wireless plans, so need to minimize data usage.
Religion and science are both 90% crap..but that doesn't negate the other 10%.
But Time Machine requires you run a Mac that can run Mac OS X 10.5, and is useless with classic,
even on a triple boot. It does not work on HFS+ volumes that have been used by 10.4, or OS 9.
Time Machine is useless to me and my client...so your primise is faulty.
No, you mean buy a RECENT Macintosh.
TimeMachine takes about 15 minutes to do the prep work
Yes, because naturally he's using a ma. He must have certainly been in a Starbucks or Panera when he posted as well. Around the Bay area nonetheless.
Join the Slashcott! Feb 10 thru Feb 17!
Did you even read the title of the submission. He wants FOSS.
Eh? Over 15 minutes? Are you backing up to an AirPort Disk rather than a wired disk? The bottleneck there would be the wireless, not your computer.
I backup a 2012 MacBook Air every evening to a 1TB 5400RPM USB drive - plug it in, it detects it, and the backup is done in 3 minutes.
1. For keeping two drives synchronized, check out GoodSync. It's powerful, and I use it to keep two separate computers holding identical copies of two major folders of data synchronized, so if one goes down, there's minimal loss of data (1 hour, max) I use this, for example, to keep a client's two 1TB collections of photos and iTunes synchronized. http://www.goodsync.com/
2. For making backups that are compact, efficient and easy to recover, look at "Disk Snapshot". It's inexpensive, robust and I've never experienced a restore failure. I make "Disk Snapshot" images of every computer, every night, in a development environment. That way, if the thing I just did breaks the system, I can restore a 100 GB Drive is less than an hour by booting from a CD and pointing to the backup on an external drive. http://www.drivesnapshot.de/en/index.htm
I'd think to use LVM and filesystem snapshots. The snapshot does the trick of journaling your changes and only your changes. You can ship the snapshot over to the backup volume simply by netcat-ing it over the network. The backup's logical volume needs to have same size as the original volume. It's really a minimal-overhead process. Once you create the new snapshot volume on the backup, the kernels on both machines are essentially executing a zero-copy sendfile() syscall. It doesn't get any cheaper than that.
Once the snapshot is transferred, your backup machine can rsync or simply merge the now-mounted snapshot to the parent volume.
A successful API design takes a mixture of software design and pedagogy.
If you are spending time messing with a system that is not going to provide you with a running computer after a quick trip to the store for a new hard drive, then maybe you should rethink your goals.
And perhaps you would regret the time spent less if you knew that in the event of an emergency, your backup would not only save your data, but prevent a re-installation and updates and more updates and more updates, and hunting for installation media and typing in software keys.
AIX had/has a nice system for backups: it created a bootable backup tape. Just turn the key, boot from the tape, say go and your machine was recovered completely. The closest I have see to that recently is clonezilla.
Making a mirror every now and again is not a backup strategy to use. This is the canned RAID is NOT a backup and never will be advice. For a single laptop something like backblaze is probably a better bet.
No sir I dont like it.
Rsync copies only changed files. The time-consuming part is reading all directories in the directory tree.
thegodmovie.com - watch it
TimeMachine will backup a running VM. It just has to backup the entire VM each time whereas with the OSX environment, only the delta changes transfer after your original full backup. To address the underlaying performance issue however, just replace the internal with an SSD. It's that bloody simple of a solution. I can run my VMs, the rest of my Mac applications and run TimeMachine all at the same time. Previously I couldn't do this with a standard HDD due to a saturation in disk I/O (hung with a spinning rainbow wheel mouse cursor) .
Life is not for the lazy.
Read from the top level and you'll see that no-one's made the assumption that he's using a Mac. This has simply become a side discussion on TM.
systemd is Roko's Basilisk.
You're holding it wrong. ;)
rsync 2.x was horribly slow as it would scan the entire source looking for changed files, build a list of files, and then (once the initial scan was complete) would start to transfer data to the destination.
rsync 3.x starts building the list of changed files, and starts transferring data right away.
Unless you are changing a tonne of files between each rsync, it shouldn't take more than a few minutes using rsync 3.x to backup a 1 TB drive. Unless it's an uber-slow PoS drive, of course. :)
We use rsync to backup all our remote school servers. Very rarely does a single server backup take more than 30 minutes, and that's for 4 TB of storage using 500 GB drives (generally only a few GB of changed data). And that's across horrible ADSL links with only 0.768 Mbps upload speeds!
Going disk-to-disk should be even faster.
two pools, internalPool, externalPool
use ZFS send and receive to migrate your data from internal to external, you and do whole fs or incremental if you keep a couple of snaps local on your internal disk, this can get excessive if you have a lot of delta or you want a long time.
http://docs.oracle.com/cd/E18752_01/html/819-5461/gbchx.html
of course you will need a system that can use ZFS, there are more options for that than time machine, its block level and its fast, and it doesn't depend on just one device, you can have multiple devices (I like to keep some of my data at work, why? my backup solution is in the same house that would burn, if it burned...)
Unix, an obscure operating system developed by bored researchers in an attempt to get a better game playing experience.
Exactly. I think rsync will do nicely. I use it for nightly backups and I rotate through 5 increments. The oldest goes to the bit bucket. Note the copy link -l option.
A snippet:
[more rotations above]
if [ -d $BACKUP_DEST/$(basename $i)/increment.0 ]; then
cp -al $BACKUP_DEST/$(basename $i)/increment.0 $BACKUP_DEST/$(basename $i)/increment.1
fi
rsync -av --delete --exclude-from="$EXCLUDE_LIST" $i/ $BACKUP_DEST/$(basename $i)/increment.0/
touch $BACKUP_DEST/$(basename $i)/increment.0
done
echo "Backup Complete on "$(date)
TM could be doing 15 minutes of work on your own HD before it bothers spinning-up the external, you realize.
You may be correct, but your evidence doesn't match your assertion.
Comment of the year
Ubuntu documentation lists some alternatives: https://help.ubuntu.com/community/BackupYourSystem one that is not listed there, and that I used many years ago is "UNISON"... I found it faster and better then rsync, also it does binary diff, so for big files that only "metadata" changed, it transfers faster.
BtSync is not officially opensource, but authors are considering opening it: http://forum.bittorrent.com/topic/17782-bittorrent-sync-faq-unofficial/
Specifically how DRBD handles recovery after an outage of the replication network. The situations where the disk isn't plugged in will look just like the network outage scenario DRBD handles. I'm not sure whether this will be more or less efficiency than the mdadm bitmap approach outlined above, but those are the two main ways people do this specific operation.
Not free but reasonably priced and worth every penny. rebit.com keeps track of all changes, sends new versions of a file to a local harddisk, a network share or both. In case of a crash you can recover from a boot CD-ROM and I've used that to transfer my files to a new computer too. They have cloud-enabled versions too.
To Terminate, or not to Terminate, that's the question - SCSIROB
I usually hate making posts where I am questioning the questioner, rather than providing an answer but with 1 TB of information you should put on the patience cap. It will take as long as it takes.
To break down what you are wanting:
I want a backup based on a journal file system sorta of thing that works incrementally slowing down every disk operation by a few milliseconds so I can shave 15 minutes off of a backup procedure, but I still have to send the same data. I don't think that would be very wise. The best existing method is to use mirror a volume but you're still experiencing the same "15 minutes" of delay.
The best thing you can have is a "fire and forget" procedure where you can walk away and let it run.
locate (based on updatedb) does not capture/sort on file modification dates so you are going to be left with a recursive file system search no matter what.
You could use find to generate a list of files that have been modified since a certain date and then feed that to tar. That way you can pre-generate an incremental backup in a file that you can copy over. Then let whatever backup solution you like make a full backup from time to time. You can setup a script that would run a few times in your work day to generate the file so at least every 24 hours there is a tar file you can copy over when you get a chance.
Good luck!
"Don't fear death... fear not living..." -me
$ rsync -av --delete
sent 1001882570 bytes received 106527 bytes 1775002.83 bytes/sec
total size is 527398084971 speedup is 526.3
It took 10 minutes to scan the 500 MB partition (ext4). From the other posts, I guess the duration time is due to the number of files that is scanned by rsync.
Btrfs send/receive should possible be doing the trick. After first cloning the disk and before every subsequent transfer create a reference-snapshot on the laptop and delete the previous one after the transfer.
$ btrfs subvolume snapshot /mnt/data/orig /mnt/data/backup43 /mnt/data/backup42 /mnt/data/backup43 | btrfs receive /mnt/backupdata /mnt/data/backup42
$ btrfs send -p
$ btrfs subvolume delete
I havn't tried this for myself, so the necessary disclaimer: this may eat your disk or kill a kitten ;-)
Time Machine keeps an event store journal of changes, the process is described at How Time Machine Works its Magic. What you're describing might be a "deep scan" pass. It's also possible you're touching a lot of directories with updates, which makes the optimization they apply not as useful.
There are cases where the event store makes Time Machine backups nearly instant, which is never the case for rsync based approaches being complained about here.
Btrfs has tools for doing this. It also comes with find-new that allows to find exactly which files have been changed between snapshots, and it does it basically instantenously.
Though Btrfs might not be the solution for ensuring data integirity at this point.. But setting up hourly snapshots of your drives can be quite nice when you accidentally destroy something you've created after the last backup.
Yes, something more recent than 2004.
What are you doing that means you need to keep OS 9 and machines older than 9 years running?
(FWIW, I don't think the OP has this problem, if he's got a laptop with a 1TB internal disk.)
Sorry, but something doesn't sound right here. Even with the amount of data your g about, it sounds like something else is slowing it down. Namely, system/kernel config.
Do some I/O observing if you haven't. You may have some task or process priority problem slowing things down.
Just a thought...
TimeMachine takes about 15 minutes to do the prep work before it starts copying for me
Well with my 2012 non-Retina MBP with a 1TB disk it only takes a few minutes at most. I guess those extra screen pixels must really slow it down! ;-)
when has a typical user ever planned ahead?
Stating the files on filesystems that require that is usually orders of magnitude more time consuming than the actual directory reading. IMO, filesystems should store mtime in the directory entry and readdir calls should return it.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
I use iFolder for this. It has clients for Windows, Linux, and Mac platforms, and works reasonably well. The server was a bit of a pain to get set up though. It used to be a Novell product but has spun off as its own open source project. You can check it out at ifolder.com
It is relatively straightforward to use DRBD to loopback across 127.0.0.1 and protocol A to set exactly this up. I'd link to the blog which demonstrates how I did this, but don't want to get slashdotted. (Googling "drbd-for-ssdusb nz" should hopefully show you how to do this if you care enough to google for it)
I think the slowdown is from the hard links it creates in the backup directory on the external drive. That takes a lot of time. Every file that's changed gets written to the backup directory as a new file. Every file that hasn't changed gets written as a hard link to the inode of the original backup of that file. So if you have 200,000 files, and 10 of them changed, you still have to write 200,000 entries for the backup.
Still - I don't ever see 15 minutes. I'm curious what's causing your problem and wonder if just blowing away your backup and starting over wouldn't help.
The Airports are really flaky for TM backup. The bottleneck I've seen with them is that they just quit working and need to be reset. Even over Ethernet.
As a mac owner, I'm sure you realize that mention of it being Retina is only related to Apple not using model numbers in almost all of their documentation and sales pages (except maybe in the fine print).
Welcome to the future. We can even use variable-width fonts now.
Do you know anything about hard-links? Hint, you have multiple directory entries pointing to the same file (inode).
It performs actions on file changes. It can also create a list of updates/created/deleted files on the fly.
Then, you just have to use this list to sync juste the changed files.
Put your filesystem on a DRBD device?
You'll get a consistent clone every time you leave your laptop connected to your home network long enough for it to sync the changed blocks
TimeMachine takes about 15 minutes to do the prep work before it starts copying for me, on a 2012 Retina MBP with 16Gb of RAM and only 256GB of disk space ... 64 GB taken by an unbacked up BootCamp part and another 120 or so eaten in Windows VMs that don't get backed up either ... i.e. Its not a slow spinning platter backing up a terabyte of data.
I see no indication of any Journal, it certainly isn't making it faster. Pretty freaking slow actually.
To what are you backing up and how much data do you generate in a backup interval? It sounds like you're backing up to a network storage device on a wireless network or just a SLOW network, OR you are generating 100s of megabytes if not gigabytes of data during a backup interval. Basically, something is either very wrong or you are a data hog for an SSD equipped machine to backup that slowly.
Do you modify all your research work from the last 20 years? If not, exclude it from backup, since you already have it backed up and are not changing it.
However, he'll want to keep in mind that, depending on his environment, he may have some other issues. For instance, I'd like to use it at work, but I can't because file access times are important to us, and rsync changes the access times on the source files. Last I checked, there was no option to make it stop that, so I'm stuck with tar.
Sit, Ubuntu, sit. Good dog.
Real backup and since 6.3 does journal based backups for Ext2, Ext3, Ext4; XFS, ReiserFS, JFS, VxFS, and NSS.
The other option I have seen (surprisingly for GPFS as TSM does not do journal based backups for GPFS even though both are IBM products) is to register to the DMAPI (this would only work for XFS I think) and then use that to capture all activity on the file system. You could then use that to generate your list of files to backup. Admittedly this is going to require you to get your hands dirty and do some coding. I am also not sure what state DMAPI support is in XFS either.
I bet you don't back up very frequently, and Time Machine determines that the record of files modified kept by FSEvents is stale. That would force it to do a deep scan, i.e., it traverses the whole directory hierarchy to figure out what has changed, much like rsync does.
If you back up every couple of days the whole backup including prep time should take under a couple of minutes. That's particularly true if you keep the default functionality of Time Machine (that is, backing up every hour).
No, when zieroh says "Time Machine fires up in the background, does its thing, and then stops shortly thereafter" he is talking about the Time Machine icon spinning around in the menu bar. That will happen throughout the whole back up process, including the prep.
I'm pretty sure that the difference is that zieroh back up very frequently, maybe using the default functionality (i.e., backup every hour), while BitZtream is more like most of us and backups every few weeks (or when he decides he has accumulated so many changes that losing them would really be painful).
No, it does not. If you read John Siracusa's excellent OS X Leopard review... oh, wait, you are the same guy. Nevermind.
With respect, there are machines running OS X where it's not worth putting recent versions that have time machine. Not all Macs have Intel.
Why don't you try "--link-dest". It's pseudo-incremental, that is: unchanged files are hardlinked to the previous backup, meaning that there's no space or bandwidth consumption for unchanged files, but each day's replica is a full backup.
Come on, this is 101 stuff, though very entertaining to see the usual geek overkill suggestions of RAID mirrors, ZFS pools, and other fun stuff. We sure love our overkill, huh? (I'm guilty too)
Anyways, what I would do is 1st ask myself "why am I watching a backup run?" Whether it takes 5 minutes or 50 minutes is irrelevant; who watches their backups run? You log the job output and email the results, or whatever your favorite technique is.
Second, set up your daily cron to launch a shell script a little before the earliest time you expect to arrive home. In this shell script you insert a simple loop that does a check to see if you're at home (more below); if you're not at home, the loop sleeps for 10 minutes (or whatever) and runs again. Set an upper bound on the cumulative total amount of time to sleep (6 hours or so); in the event you're out really late/all night, then the script exits cleanly without performing the backup and runs the next day. It can identify you are at home and ready to launch the rsync by one of several conditions: look for the presence of an NFS mount; look for a certain IP range; look for your USB drive mounted that perhaps has a file named ".backup_device"....whatever. When this condition is TRUE, break out of/exit the loop, and the next line contains the rsync command. Easy!
One of the most important aspects of a backup job is that it is scheduled and runs automatically. If you have to rely on a person to manually start it, then sooner or later (like, within a week) you'll start putting it off or simply forget to do it.
Tar does it. Why can't rsync? Sorry, but that makes it pretty much useless for backup (in all the cases I have to work with), and most the other IT people with whom I've discussed this agree.
Sit, Ubuntu, sit. Good dog.
The agency where I work is all Microsoft, all the time. They use Active Directory for everything. After I had been working there a year, security did a scan of the network and found out I was running Linux. I ran Linux for a year in that environment and had no trouble. My Linux desktop played just fine with their AD forest. LibreOffice had no trouble with any of the Microsoft Office documents I needed to handle. I've heard that some version of LibreOffice had some trouble with some feature as implemented by some version of MS Office. I saw no problems, though.
I did get in trouble for running an unauthorized OS. Now I'm still at a command line, but on a Mac when I need to be connected to the office network. Mac is certified Unix.
That's a great solution, and one that actually answers the OP's question, assuming he can do a little shell scripting. I spent YEARS designing a great enterprise grade backup solution using LVM. I'm the maintainer of Linux::LVM, but I hadn't heard of lvmsync before. I'm sure I'll use it sometime.
Just curious, why do you require access time? I set 'noatime' on all partitions.
I use BackInTime under similar conditions and it works fast and perfect. It uses diff, rsync and symbolic links to preserve the history of every snapshot. Check it out: http://backintime.le-web.org/
IMO, wherever hard links aren't, inodes should be inlined into the directory entries and read and stored in cache whenever the directory entry is read. Hard links make up a small percentage of files, especially in typical large-scale storage systems. Inlining inodes should solve a lot of unixy performance issues, but retrofitting arbitrary placement of inodes into filesystem code is prohibitively difficult because unix expects inodes to be accessible independant from a directory entry. A nice middle ground might be making inlining inodes a filesystem creation option which disables hardlinking on that filesystem. This is obviously doable since NTFS can be mounted on Linux.
Another nice option might be to add a separate piece of metadata which would be changed whenever an inode or the file it's attached to change that's attached to the directory entry. This way, a scan of a filesystem for changes could be quickly thorough. Essentially a directory stored mtime.
set softtabstop=4 shiftwidth=4 expandtab nocp worlddomination
https://wiki.archlinux.org/index.php/Backup_Programs
Casteism