Slashdot Mirror

git-annex and Amazon glacier might serve you well. Encrypting your GIT/Glacier archive using your PGP key is a one-click-and-save option. With Google's recent announcement of Nearline I imagine over time it will be supported also. GIT annex came about through a kick-starter campaign, and you're welcome to support the project.

Here's some links to help you:

http://git-annex.branchable.co...

Specifically for Glacier:
http://git-annex.branchable.co...

git-annex and Amazon glacier might serve you well. Encrypting your GIT/Glacier archive using your PGP key is a one-click-and-save option. With Google's recent announcement of Nearline I imagine over time it will be supported also. GIT annex came about through a kick-starter campaign, and you're welcome to support the project.

Here's some links to help you:

http://git-annex.branchable.co...

Specifically for Glacier:
http://git-annex.branchable.co...

You might be interested to know about git-annex then. Here's some links to help you

http://git-annex.branchable.co...

Specifically for Glacier:
http://git-annex.branchable.co...

You might be interested to know about git-annex then. Here's some links to help you

http://git-annex.branchable.co...

Specifically for Glacier:
http://git-annex.branchable.co...

Here's an 'unofficial' open-source bit-sync client:
www.yeasoft.com/site/projects:btsync-deb:btsync-server

It doesn't install on .rpm based distros so far as I can tell. I have a use-case that calls for drop-dead-easy cross-platform sync, and I'm leaning towards using git-annex assistant, but haven't had time to thoroughly test it yet.

I was thinking along your lines, instead of getting rid of all the video, though, I think people would get back to the habit of downloading and sharing stuff via local networks or sneakernet, and discover ways to keep local storage in sync like with git annex. Also, online content that is DRM and stream-only would suffer.
But this would make some big interests very upset, and Hungary government is no match for them, so they will find some way to curb alternative uses of networking. This means that we won't see again the chaotic but decidedly more interesting landscape of 20 years ago (last days of BBS, usenet, internet protocols instead of web based centralized megasites).

Git Annex ( http://git-annex.branchable.co... ) (if you're a geek) is the perfect answer to maintaining multiple copies of digital data.

6: An Amazon Glacier client for archiving documents for the long haul.

May I refer you to git-annex-assistant?

http://git-annex.branchable.com/

git annex is an open source project that lets you distribute files around various media (including external HDs, Amazon S3, SSH-connected computers, etc.). It has an fsck command for checking that your data still matches its checksums.

There's a GUI interface that makes it a lot like Dropbox, where you just add files to a folder, and they are sync'd.

It works on OS X and Linux, with an alpha for Windows.

git annex is an open source project that lets you distribute files around various media (including external HDs, Amazon S3, SSH-connected computers, etc.). It has an fsck command for checking that your data still matches its checksums.

There's a GUI interface that makes it a lot like Dropbox, where you just add files to a folder, and they are sync'd.

It works on OS X and Linux, with an alpha for Windows.

git annex is an open source project that lets you distribute files around various media (including external HDs, Amazon S3, SSH-connected computers, etc.). It has an fsck command for checking that your data still matches its checksums.

There's a GUI interface that makes it a lot like Dropbox, where you just add files to a folder, and they are sync'd.

It works on OS X and Linux, with an alpha for Windows.

Git Annex ( http://git-annex.branchable.com/ ) is awesome at doing it's syncing when it's possible and has a somewhat nice web UI over great CLI tools. It can also do encrypted backups which may be important if you don't fully trust your neighbour.

I've tried SparkleShare and it works really well, so long as you don't have many large binary files, like images or videos. It fails where traditional GIT fails.

What I found that works better is git-annex assistant and either your own redundant and cheap hardware disks, or you can also ssh somewhere, OR you can also use Amazon Glacier for a very very low cost. Yes, you can also encrypt everything before it leaves your machine. Check out the nice video tutorials.

http://git-annex.branchable.com/assistant

You might want to checkout git-annex: http://git-annex.branchable.com/

It handles the idea of larger repositories with disconnected parts. You get git versioning of files and the ability to replicate portions of the data at will.

I second git. git scales badly with large files. If this is a problem, you could have a look at git-annex http://git-annex.branchable.com/ The concept needs some time to grasp, but it's really powerfull.

For something Dropbox-like in UI that you can point to your own servers, some options are:

* Git-Annex Assistant: Despite its name, git is sort of an implementation detail you can ignore. It doesn't actually revision-control all your files, so you don't get huge bloat with binary files that are edited. One nice thing it does is integrate syncing with offline storage, so you can e.g. set up a remote server to sync to live, *and* set up a USB-connected hard drive to sync to when it's attached. When the USB drive is offline git-annex will still remember what files were on it.

* Sparkleshare: a front-end that does version-control all your files, which might be preferable if you are sharing small-ish files where you might want to recover a previous version (e.g., text documents). Less good than Git-Annex Assistant if you're sharing huge media files, possibly better if you aren't.

See also this Slashdot discussion from two years ago.

git-annex (and git-annex assistant for those that don't like CLI):

    https://git-annex.branchable.com/assistant/
    https://git-annex.branchable.com/

does lots more than just sync you files, and can do that with proper encryption (GPG) to a load of cloud providers, or to your own servers, or without needing a server at all. I could go on, but it would be better to just follow the links.

git-annex (and git-annex assistant for those that don't like CLI):

    https://git-annex.branchable.com/assistant/
    https://git-annex.branchable.com/

does lots more than just sync you files, and can do that with proper encryption (GPG) to a load of cloud providers, or to your own servers, or without needing a server at all. I could go on, but it would be better to just follow the links.

There are a lot of friendly projects fwiw, and many love any kind of new contributor, of any skill level, whether they're submitting technical contributions or documentation or even just comments on common use-cases that worked for them or didn't. Kernel development is probably the worst place to start, for a variety of reasons.

I've recently had good experiences interacting with the Racket and git-annex maintainers, to pick two examples.

You might want to take a look at git-annex. It lets you manage large files, and also use backends like S3 for storage.

I manage my files using git-annex which supports a number of storage services with seamless GPG encryption.

I manage my files using git-annex which supports a number of storage services with seamless GPG encryption.

"go easy on the 'just apt-get FubarPackageInstaller.gzip and rd -m Arglebargle' stuff"...

So why is he asking Slashdot?

Seriously, git-annex assistant might be the solution for part of his problem. assistant is a pretty front end to git-annex which uses git to sync repositories of file metadata and several other means (rsync, etc) to schlep the files between repositories. It won't run on his Windows box but does run on OS X and might be easy enough for his wife to use.

"go easy on the 'just apt-get FubarPackageInstaller.gzip and rd -m Arglebargle' stuff"...

So why is he asking Slashdot?

Seriously, git-annex assistant might be the solution for part of his problem. assistant is a pretty front end to git-annex which uses git to sync repositories of file metadata and several other means (rsync, etc) to schlep the files between repositories. It won't run on his Windows box but does run on OS X and might be easy enough for his wife to use.

You might want to look into git-annex:
http://git-annex.branchable.com/

I've not tried it, but it sounds like an ideal solution for your request, especially if your data is already compressed.

You might be interested in git-annex ( http://git-annex.branchable.com/ ).

After you set up some repositories, you can say "git annex get my_home_movie.avi," and it will figure out which of your repositories has it, and copy it over for you. (It also checks that a file exists in another location before it lets you delete it).

As far as I can tell, the use-case it was designed for was one user with multiple places to put files. It might be pretty cool to extend it to work well with many users. (Although at this point, maybe we're just reinventing eMule et. al).

If you are using Linux or *BSD (including OS X), git-annex is the way to go. Seriously.

http://git-annex.branchable.com/
http://www.kickstarter.com/projects/joeyh/git-annex-assistant-like-dropbox-but-with-your-own

You seem to have access to a website you could already publish it on no?

Failing that for whatever reason you could put it in a wiki on branchable? No I'm not affiliated to them in any way but they were the first "good" answer which jumped to my mind.

More obscure but perhaps extra appropriate for the topic at hand, you could publish it on a "hidden service" on tor?

Thanks for git-annex. At least one new answer to an old problem.

There is a problem that the mainstream cloud storage options like DropBox, SkyDrive, MobileMe, Google Music, etc. all store your data unencrypted, meaning eventually the MafiAA will sue your asses based upon the media that you've archived there. Wuala encrypts documents using the document's own SHA as the symmetric key for deduplication, meaning they cannot read your documents, but any MafiAA like party can still identify your documents.

Afaik, you still need command line tools like duplicity, git-annex, and jgit for encrypted cloud storage via Amazon S3 or others. Syncany might fix this.

Imho, we need an encrypted cloud storage solution that is resistant to even traffic analysis and offer social functionality. It might resemble the following :

Layer 1. Anonymized ad serving and/or payment via digital cash systems : An advertiser gives you a coin when your app claims you've showed an ad, you anonymize that coin and give it to the hosing provider, hosting provider redeems coin with advertiser, adjusting their payout based upon the advertisers identity. Ideally, the hosting provider and advertiser shouldn't be able to trace their relationship to you unless they violate the protocol by comparing IP address, which you may defeat by using a trusted anonymizing bank. Anonymized payments could be are handled similarly but might create issues with banking laws if the coins represent real currency. Tor, I2P, and Freenet could also use this layer help their users earn money.

Layer 2. Anonymized automated bitlocker based storage : Your application creates a 'thread' on a host by uploading a 8192 bit RSA public key, creating a symmetric AES 512 key to save alongside the private key. Threads contain three types of messages : unsigned public messages that applications will ignore unless they're encrypted using the symmetric key, signed public messages that may be unencrypted, like maybe deleting an old message or closing the thread, and private hello messages that applications will ignore unless they're encrypted using the private key. Hosts are federated allowing users to submit their signed messages through other hosts to prevent their preferred host from identifying the thread owners IP address.

Threads are identified by their public key's SHA512. You may grant anyone read & 'reply' access to a thread by giving them the threads id and symmetric key. You may hash identifying information like your real name or email using SHA512 and submit that plus a thread id to lookup servers. You're real threads should NOT however be available for lookup. Instead, your application replies to hello messages by sending some real thread, ala work, family, whatever.

Oh, all thread content is accessible by anyone, all privacy is accomplished through cryptography. It's actually a feature that all this data becomes public once quantum computers can break 8192 bit RSA keys, which'll happen long after your dead.

Layer 3. You're application provides a 'social versioned file system' using a hosting layer thread or ten and pays the hosting provider using the ad serving layer. Imho, the underlying file format should be packed git repository extended to offer quasi-instant messaging attached to objects, roughly like github's comments.

End result : People archive their photo, video, music, etc. collections online, grant their friends access, and chat with their friends in instant messages affiliated with the files, roughly like facebook comments. Of course, the whole system works perfectly for collaborative private projects, like university homework assignments. All users are just some collection of threads they control but nobody knows what threads do what.

http://www.branchable.com/ (or acutally ikiwiki) is a wiki that you can work on offline. And there's vim-outliner if you don't like emacs orgmode.

git-annex uses git to track your metadata and rsync to move your files around. It knows which repos hold what files and can enforce minimum copies, trust levels, etc.

Also, it can store transparently encrypted data with untrusted third parties like Amazon S3. You can even have it use bup as a back-end which gives you change tracking of actual data, not only metadata. Oh, and a FUSE front-end is in the works which means you get 100% transparent file tracking, distribution and backup. All based on FLOSS and you are in control.

If you know how awesome VCS are and want to use them to actually get some order into your files, configs and maybe even life, click the links below.

http://git-annex.branchable.com/
https://github.com/apenwarr/bup
http://lists.madduck.net/listinfo/vcs-home
#vcs-home on irc.oftc.net