Slashdot Mirror
Laptop/Server Data Synchronization?
gbr writes "I've been trying to automatically synchronize data between a laptop and a server. When the laptop is connected to the network, I want all writes to automatically propagate across to the server. When the laptop is disconnected I want the laptop user to continue working with the local data. When the laptop is reconnected, I want the data to automatically re-sync.
The issue is, the data on the server may have changed as well, which needs to propagate back to the laptop. The data doesn't contain anything too special, no database tables etc. It does contain binary data such as executables and word processing documents. I've looked at ChironFS, Unison file sync, and drbd. ChironFS needs a manual rebuild if a connection fails, and the user needs to know which machine contains the correct data. Unison requires the user to initiate the synchronization process manually every time, and drbd is just not meant for the job at hand. How do you automatically, and invisibly to the user (except in the case of conflicts), synchronize between a laptop and a server?"
My Briefcase in Windows 95. It even has a cute ickle briefcase icon.
Somewhat seriously, Offline Files in Win2K/XP is something I've yet to see done well on any other OS.
neuro at well dot com (when I post, it's my opinions, no-one elses)
At the risk of saying something stupid or blasphemous: why offer something that requires "writing some scripts"?
If the OP wanted to "write some scripts" s/h/it could have done all the neccessary work with a couple foreach...cp...end. Or, hey, rsync.
I am suspecting the OP is wondering whether there isn't something out there that "just kinda works" and only needs intervvention in case of a conflict.
Knowing well that this will definitely be considered blasphemy: I've been using Window's "briefcase" system since Win98. It does "kinda work". Most of the time. And requires work when there's a conflict. Which appears to be what the OP is looking for. Given that the OP doesn't seem to want to just go that route, the question appears pertinent what s/h/it is looking for that Mr. Gates briefcases can't/won't do...
We're all born with nothing.
If you die in debt, you're ahead.
Offline Folders on a Windows client connected to a Windows server work reasonably well but sometimes get screwed up.
Novell's iFolder is a very interesting alternative... runs on Linux/Apache/Java stack & only transmits changed blocks over an SSL connection.
Other things worth looking into include Microsoft Groove--let's you synchronize an entire workspace with yourself on other computers or other people - and is relatively network & environment-independent (though Windows only)
>What do you mean if iFolder was mature?
I don't know how mature the novell version is but the open source version is very far from being mature. In fact there hasn't been a stable release in more then two years and nobody knows if or when there will ever be a stable release.
A while ago all the developers on ifolder either quit or were fired and the development was moved to india. Since then the pace of development has slowed down to a crawl and the new developers try to understand the code base and fix bug reports.
Right now you can download something that is beta-ish but I certainly would not trust my mission critical data to it. If you want something that works you are going to have to pull off the trunk and compile it.
evil is as evil does
A few people hit this one pretty well. rsync (and probably rsyncd).
:)
The more complex problem has been thrown at me a few times. What if it's not just one person?
Say you have a repository of data that a dozen people may be working in. When they're all network connected, they're all dealing with the same file pool. When they take their off-line copies with them (unplugged laptops on vacation), they all make changes to the same files. Maybe mine is a one line change. Maybe one guy copy&pasted the first 3 chapters from War And Peace into a comment somewhere in the middle. Maybe another developer did some very intellectual looking changes but hosed some major functionality.
When you start putting machines back on the network, who is right? The 6 guys who did real work are obviously right(ish), but they all made different changes. The very last change will end up being someone's 3 year old kid who was pounding on the keyboard right before daddy shut down the laptop, saving the new changes. Probably the last is the most recent, and right by most methods.
It's not a pretty picture, and requires some intelligence to sort out the mess.
The only "good" resolution I've found is to give logical authority to the changes. Bob is in charge of development. Any changes going into the development or production tree must clear him. He should be able to recognize that the 6 guys made changes, and diff them to come up with the common changes. The 3 chapters of war and peace go by the way side. And the guy with the 3 year old "developer" gets reprimanded.
In the end, a good revision system and good backups are needed too. Something will slip through the cracks, and you'll need to roll back to something you hope is good.
I take control over whatever I'm working on, so if I know I'll be working offline, I'll scp the data to my laptop, work on it on the road, and scp my changes up to the server when I'm done. Anyone else who may have worked in my project space in the duration should have known better.
Serious? Seriousness is well above my pay grade.
Apologies if it's in bad taste to reply to my own post, especially because it's about the product I'm working on, but here are some of Dropbox's differences/improvements over what people typically hack together themselves:
r -list/
- syncs continuously/watches the FS for file changes (no cron jobs needed -- things usually sync as quickly as they can be sent)
- does binary diffing and only sends deltas (compressed & over SSL)
- transparently archives past versions of all files (i.e. undelete/infinite undo)
- syncs across any number of machines
- lets you get to your files from the web
- some more info @ http://venturebeat.com/2007/08/16/the-y-combinato
We made it after hacking together our own rsync-based abominations and getting really annoyed that no one had solved this genre of problems in a way that normal people could use.
Okay, I can stop shilling now. I was just excited that other people run into these problems.
"Where are we going, and why am I in this handbasket?"