Slashdot Mirror

I'm very late in the stream here, and I also know it's quite overused to start a comment with "I am surprised not seeing product XYZ" ;-)
But still, come on, nobody here mentioning shared, end-user-based storage like for instance https://tahoe-lafs.org/trac/ta... ?

My understanding is as soon as you can provide a machine permanently connected to internet, offering a given storage volume, the very purpose of systems like Tahoe-lafs is to provide open-source, shared, encrypted and redundant storage -linux-based... And they are very active...

I think this is a major part of TAHOE-LAFS. https://tahoe-lafs.org/trac/ta...

Tahoe-LAFS is a system that helps you to store files. You run a client program on your computer, which talks to one or more storage servers on other computers. When you tell your client to store a file, it will encrypt that file, encode it into multiple pieces, then spread those pieces out among multiple servers. The pieces are all encrypted and protected against modifications. Later, when you ask your client to retrieve the file, it will find the necessary pieces, make sure they haven't been corrupted, reassemble them, and decrypt the result.

Just one word: Tahoe-LAFS. Deploy, use, relax.

A small plug for Tahoe-LAFS.

It doesn't matter where it is. It uses cryptography to give you what you want. Mirror in many places including on your own machines for redundancy.

https://www.tahoe-lafs.org/tra...

For client-side encrypted backups, duplicity is popular, however it is a bit too buggy in edge cases for my taste.
dar is similar but seems more mature, however takes up more space (deduplication is less efficient than Duplicity) IIRC.
For offsite online backups you can check out: Tahoe-LAFS, Obnam, tarsnap, brackup, attic.

Give a check to Tahoe-lafs , that could be what you are looking for.

From their page

What we mean by "security" is something different. The service provider never has the ability to read or modify your data in the first place: never. If you use Tahoe-LAFS, then all of the threats described above are non-issues to you. Not only is it easy and inexpensive for the service provider to maintain the security of your data, but in fact they couldn't violate its security if they tried. This is what we call provider-independent security.

Give a check to Tahoe-lafs , that could be what you are looking for.

From their page

What we mean by "security" is something different. The service provider never has the ability to read or modify your data in the first place: never. If you use Tahoe-LAFS, then all of the threats described above are non-issues to you. Not only is it easy and inexpensive for the service provider to maintain the security of your data, but in fact they couldn't violate its security if they tried. This is what we call provider-independent security.

Lavabit and silent circle inspired me to think about some kind of peer to peer distributed email system.

Although currently everyone can install an email server (e.g. there are several available in debian). It is not what would solve the problem. Not just because it requires technical expertise, but also because it requires too much dedication on your side to maintain your freshly installed server. Also to make sure it has outside access with SMTP port, and so on. Not mentioning that it needs about 100% uptime. Such solution is too much centralized.

I was thinking about p2p email more like this one which I googled right after I had this initial idea. This is a proof of concept so it can work.

Key features would be:
1) uses p2p distributed encrypted file system, like tahoe
2) each p2p node can act as email receiver/sender
3) to send email to someone you use nick@1.2.3.4 where 1.2.3.4 is any IP that is running p2pemail. Simplest would be 127.0.0.1 if you just run a p2pemail node yourself.
4) everyone can have p2pemail account, just connect via https to nearest p2pemail node. It can be running on your computer or anywhere else. Doesn't matter. This just requires setting up an account name on your side, and a lenghty password, which is also used as a sha256 seed for private key for encryption of your emails and also as a PGP signature for you emails.
5) PGP signing emails would be so easy, that it would be a new standard.
6) all encryption and decryption is done locally on your computer either in javascript or in your email client. Just make sure that your browser and computer are not compromised.
7) if any of p2pemail nodes are running compromised code (eg. like compromised tor nodes) they still cannot read your email, because they have no acces to your private key. The only hope they can have is to monitor when you are accessing your data, but only if a request to the compromised node is made.
8) even if huge NSA datacenter decided to store all p2pemail data, they still cannot read it, and have nobody to file a warrant to.

If we combined that with bitcoins we would get additional (optional) features:
9) buy storage with bitcoins, while buying decide how many copies of your data you want to have (can change this anytime later). Offer any price you want, lower bids might not be taken.
10) provide encrypted storage space and get paid. If you store multiple copies of same data (might be possible before p2pemail gets popular) ensure that at least it is on different physical locations, otherwise you might be compromising security
11) create whitelists with people from whom you want to receive email, add mandatory bitcoin fees if anyone not on the whitelist wants to send you email.
12) You can create various stages if whitelisting, depending on domains you can define different prices to receive email. Or you can say that first email is free for everyone, and each next will be paid or not depending on if you received spam. Or configure spamassasin to decide for you.

PROBLEM: where do my friends send email to?
ANSWER: your_nick@p2pemail.org/net/com/info (we need to register many domains, and use many IPs to resolve those dns-es)

PROBLEM: Will my address still be the same after long time?
ANSWER: your nick in p2pemail will be the same, tell your friends that if they cant send email (eg. govt seized all p2pemail domain names), then they have to find some p2pemail node. Google it, or install one themselves. If they can't do that, you can solve this by installing a node yourself, and making sure it has the same domain name all the time. Services like dyndns can help you with that.

well maybe that's just a pipe dream. But the proof of concept implementation that I linked above gives some hope. What do you think?

Tahoe-LAFS may be something like what you're after:

The first, and most interesting, is Tahoe LAFS. It does come with a FUSE driver, so it can be mounted like a regular filesystem. It is cloud-based and redundant to a degree you choose yourself. All copies stored are encrypted, so the only person who can read them is you. I'm not sure though if fetching from more nodes than you strictly need to reconstruct your original file actually buys you anything with that system, but I think it does.

You could also use something like a mountable version of Google Drive and then layer fuse-encfs on top of it. That's not quite as secure as encrypting at the block layer. The overall shape of your directory hierarchy is available, even if the individual file names and their contents are obscured. That should probably be good enough for most purposes.

The first, and most interesting, is Tahoe LAFS. It does come with a FUSE driver, so it can be mounted like a regular filesystem. It is cloud-based and redundant to a degree you choose yourself. All copies stored are encrypted, so the only person who can read them is you. I'm not sure though if fetching from more nodes than you strictly need to reconstruct your original file actually buys you anything with that system, but I think it does.

You could also use something like a mountable version of Google Drive and then layer fuse-encfs on top of it. That's not quite as secure as encrypting at the block layer. The overall shape of your directory hierarchy is available, even if the individual file names and their contents are obscured. That should probably be good enough for most purposes.

How about Tahoe-LAFS?

By the way, it has a too hard name -- every now and then I want to mention it but keep forgetting what Totse-TANSTAAFL was it again!

You might want to try tahoe-lafs if you want to share stuff with a fair number of people without giving them default access to the content.

I use a secure distributed grid. The software is an open source tool, Tahoe LAFS (http://tahoe-lafs.org). The grid is composed of ~15 servers contributed by different people all over the world. There are a half dozen servers in various locations in the US, about the same number in Europe, and the remainder in Russia and the Ukraine.

My files are AES256-encrypted on my machine, split into 13 pieces using Solomon-Reed coding, any five of which are sufficient to reconstruct my files, and then those 13 pieces are distributed to the servers in the grid. I run daily backups, but since uploads to the grid are idempotent, only the changed or new files are stored. I also run a bi-weekly "repair" operation which checks all of my files (all versions, from all backup runs) to see if any of their pieces are lost. If so, it reconstructs the missing pieces and deploys them to servers in the grid. The individual servers in the grid are fairly reliable, but problems do happen, so repair is important.

I get about 100 KBps net upload rate, so this isn't a good solution for backing up terabytes, and the occasional "surge" in my data generation (usually caused by a day of heavy photo-taking) often causes my "daily" backup to take a few days to run, but all in all it works very well.

Should my server ever die, I only need two pieces of information to get all of my data back: The grid "introducer" URL, which will allow me to set up a new node connected to the grid, and my root "dircap", which is a ~100-byte string containing the identifier and decryption key for the root directory of my archive. That directory contains the decryption keys for the files and directories it references.

Since this grid is all volunteer-based, the only cost to me for this backup solution is the hardware and bandwidth I provide to my grid (I provide 1 TB of disk and grid usage consumes a fairly small fraction of my Comcast connection), plus the time I spend administering my server and checking to see that my backup and repair processes are running. Oh, and I also contribute (a little) to the Tahoe LAFS project, but that's due to interest, not a requirement.

I'm very, very happy with this solution.

BTW, the grid could use another 20 nodes or so, if anyone is interested. There's a fair amount of trust required of new members to the grid, though, so it might take us a while to vet new members. The trust is required not because other members of the grid might have access to files that are not their own, but we need to verify that new members will behave appropriately -- providing their fair share of storage and bandwidth, and not consuming too much.

Anyone interested should check out the grid's policies and philosophy at: http://bigpig.org/twiki/bin/view/Main/WebHome. If all of that looks good, join the mailing list, introduce yourself and we'll consider allowing you to join the grid.

Personally I'm surprised that nobody has come up with an application that basically merges what TOR and Freenet does into one. A distributed storage would provide both the capacity and the upload bandwidth, while freeing up resources from onion sites.

Funnily enough, Tor arose from a project called Free Haven that aimed to do pretty much what you're suggesting: robust, distributed storage with anonymous access. So far the anonymous access part of the problem has attracted more attention than the distributed storage part, so we have a very high-quality anonymous access component (Tor), but until recently there wasn't a distributed storage component of comparable quality. Perhaps Tahoe LAFS is ready to fill that role.

Unlike Free Haven, Freenet didn't separate the anonymity problem from the storage problem - it tried to solve both problems at once. Nikita Borisov found that Freenet 0.5 provides little or no anonymity to a substantial fraction of users. As far as I know, nobody has analysed the anonymity provided by more recent versions of Freenet, which is not a situation anyone relying on Freenet for anonymity should be happy about. Tor has received a lot more analysis from the security community, and while its anonymity properties are limited, those limits are well understood.

TLDR: Combine Tor with distributed storage, but don't use Freenet.

Try Tahoe-LAFS.

I think that you are looking for redundancy at too small a scale: Yes, per-box, there is very little redunancy. RAID-6 makes it not completely useless; but a PSU going out will take out half the box, which will render it pretty useless until the PSU comes back online, and if the mobo dies, game over.

However, as the pictures suggested, they are running rather a lot of these boxes. Their (proprietary) software layer handles storing data across all the boxes and presenting it in some useful-to-the-backblaze-client way over the internet. An OSS analog would be something like Tahoe-FS treating each storage box as a backend server. In that scenario, you can, depending on the desired tradeoff between cost and risk, allow one or more entire servers to fail without compromising the overall logical filesystem...

Homepage
"Tahoe-LAFS is a Free and Open cloud storage system. It distributes your data across multiple servers. Even if some of the servers fail or are taken over by an attacker, the entire filesystem continues to function correctly, including preservation of your privacy and security."
Still in active development.

I don't know dropbox that well, but this might be something you want:

http://tahoe-lafs.org/trac/tahoe-lafs

Well, I have the knowledge, and I have designed systems that use cryptography in the past. You're right though, until I've actually done it, it's all just hot air. :-)

Though, if I were DropBox, I would've just used Tahoe. Of course, as someone else mentioned, that doesn't really effectively do de-duplication. So perhaps the hypothetical service I designed that way couldn't have worked as well.

If it doesn't, zfec should:

http://tahoe-lafs.org/trac/zfec/browser/trunk/zfec/README.rst

But it doesn't do any file verification (which if may make up for by being faster than par2, if you are prepared to do what you speak of above, adding in some hashing doesn't seem like it would be a big deal).

You've got to trust Amazon to respect the privacy of your data.

No you don't.