Slashdot Mirror

This has an additional problem, the Windows backups aren't encrypted. Not good if you have sensitive information.

<plug>Throw rsyncrypto into the mix</plug>

This has the downside of being a preprocessing step (i.e. - you need local storage for the encrypted form of the files), but solves the encryption problem better than your suggestion (which encrypts the transit, but not the actual backup).

Shachar

Spreading FUD all over, aren't we?

First, Skype is not, and has never been, Israeli. ICQ hasn't been Israeli for ages and ages (sold to AOL, that's America Online) in 1998. That's 17 years ago. Either way, a search for "ICQ snowden backdoor" shows nothing relevant in any of the first 10 results, causing me to question the validity of trusting you as a source. If I'm wrong, by all means, please do provide sources.

Second, I used to be in charge of Check Point's product security (late 2000 to early 2003). If any Israeli product is backdoored, you'd expect Check Point's Firewall-1 to be it. In order for that to work, I'd need to know about it, or I might accidentally close the back door. I give you my word as a non-anonymous long time user of this site that no such intentional back doors exist in the product. I have never been asked to not fix a problem I've found, or to not look for certain types of security problems.

During my time there, a few security problems were found in FW-1. If memory serves me right, most were in the management and not in the actual enforcement unit. Either way, I have never seen such a problem and thought "this seems intentional". They always seemed like no more nor less than the usual sloppy programming creating security holes.

Israel has a notorious "cypher law". I actually did produce an encryption product. I only registered it after several years in which it was freely available through sourceforge. The registration process included me sending a request with links to the web site, and a reply saying it was approved as a "free encryption device" (i.e. - I do not need to re-validate it unless I change the crypto).

Now, I know the usual FUD about rsyncrypto, and I know people will say that that's because rsyncrypto's encryption sucks to begin with. All I can say about that is that the cypher law makes it legal to use freely available encryption from the internet without restriction (i.e. - gpg, ssh etc.). They also list the number of applications they processed and denied, and the last time they denied any application was around 2002 (I cannot find the page right now, sorry).

So, all in all, I think this:

i never seen anything come out of israel that wasnt backdoored.. Icq skype etc
i think showden files had things about this also

is concentrated bullshit.

Shachar

To be fair, there is some confusion between people like yourself, who advocate the user's freedom to choose whether to use free only software, and the anti-GPL crowd, who advocate a developer's right to choose whether their addition are free or not.

While I am all for the user's freedom to not use free software (and, in fact, the non-free repository is enabled on my machines, and like I said, I do have some proprietary software installed), whenever I choose a license for free software that I write from scratch, I (usually, there are exceptions) choose a copyleft license.

I think the heat from the later argument is warming up the former argument, despite the fact there are few good arguments to limit a user's freedom of choice for the sake of giving her more freedom.

Shachar

To be fair, there is some confusion between people like yourself, who advocate the user's freedom to choose whether to use free only software, and the anti-GPL crowd, who advocate a developer's right to choose whether their addition are free or not.

While I am all for the user's freedom to not use free software (and, in fact, the non-free repository is enabled on my machines, and like I said, I do have some proprietary software installed), whenever I choose a license for free software that I write from scratch, I (usually, there are exceptions) choose a copyleft license.

I think the heat from the later argument is warming up the former argument, despite the fact there are few good arguments to limit a user's freedom of choice for the sake of giving her more freedom.

Shachar

<plug>Or, better yet, use rsyncrypto.</plug>

The advantage is that the incremental diffs don't accumolate on your computer, making your entire archive volatile (lose one rdiff, lose everything after that point). You just sync like you always do.

Theoretically, rsyncrypto is less secure. I am, of course, far from being objective about this point, but I believe this is not a practical weakness for most people, even with the renewed (justified) paranoia. Then again, the tradeoffs are clearly discussed on the project's site, so you are free to draw your own conclusions on the matter.

Shachar

If you contacted me and reported a bug in fakeroot-ng or rsyncrypto, I'd fix it. I'd do it for free. I'll say "thank you" for reporting it.

If you contacted me with the precise same bug, and offered to pay me $1000 to fix it, I'd take your money and fix it as soon as I could, because I believe it is okay for FOSS developers to make money from their work.

If you contacted me and offered to pay me $10, I'd probably be offended.* If you can't afford to pay me a reasonable fee for my time, then ask me nicely to volunteer it. Do not, however, presume to pay me an unreasonable fee for it. There are things I'd happily do for free that I will simply refuse to do for a reward that is demeaning.

Shachar

* - If you waited for me to fix it, and then contributed $10 to my pay pal account, I'd not only say "thank you", I'd even happily tell everyone I know that someone did it. $10 makes for a lousy paycheck, but it's a perfectly reasonable donation.

If you contacted me and reported a bug in fakeroot-ng or rsyncrypto, I'd fix it. I'd do it for free. I'll say "thank you" for reporting it.

If you contacted me with the precise same bug, and offered to pay me $1000 to fix it, I'd take your money and fix it as soon as I could, because I believe it is okay for FOSS developers to make money from their work.

If you contacted me and offered to pay me $10, I'd probably be offended.* If you can't afford to pay me a reasonable fee for my time, then ask me nicely to volunteer it. Do not, however, presume to pay me an unreasonable fee for it. There are things I'd happily do for free that I will simply refuse to do for a reward that is demeaning.

Shachar

* - If you waited for me to fix it, and then contributed $10 to my pay pal account, I'd not only say "thank you", I'd even happily tell everyone I know that someone did it. $10 makes for a lousy paycheck, but it's a perfectly reasonable donation.

I have over two decades of hands on experience. I am fluent in C and C++, and can get by quite well with bourne shell, python, perl and PHP. I know quite a bit of assembly for more than one platform. I have designed and coded fairly complex algorithms (e.g. rsyncrypto) and also do complex system programming (fakeroot-ng), and those are just part of the open source programs I've written.

I cannot program on a whiteboard.

I don't know what it is. Maybe it's the pressure of fitting it all in, or the bother involved in fixing stuff. Anything even remotely non-trivial, and I start doing mistakes that would make you think I am a total rookie.

A while back, I was asked in a job interview to write a reader/writer lock (one reader, one writer) using mutexes. I started coding on the whiteboard, and noticed that my solution was becoming more and more complicated, involving more and more state variables, each needing a mutex to protect it. I then stopped, and asked to do this on a laptop. I solved it in less than two minutes.

I could probably improve this skill, had I thought it was useful for anything other than job interviews. Had it been the other way around, I would obviously be useless as a programmer. As things stand, however, I think a bare minimum of keyboard and a text editor (any text editor, notepad included) is not much to ask. To date, I have always asked for one, and have always been given one, so it has never been a problem. I don't know how big companies, such as Google, would handle such a request, however.

Shachar

I have over two decades of hands on experience. I am fluent in C and C++, and can get by quite well with bourne shell, python, perl and PHP. I know quite a bit of assembly for more than one platform. I have designed and coded fairly complex algorithms (e.g. rsyncrypto) and also do complex system programming (fakeroot-ng), and those are just part of the open source programs I've written.

I cannot program on a whiteboard.

I don't know what it is. Maybe it's the pressure of fitting it all in, or the bother involved in fixing stuff. Anything even remotely non-trivial, and I start doing mistakes that would make you think I am a total rookie.

A while back, I was asked in a job interview to write a reader/writer lock (one reader, one writer) using mutexes. I started coding on the whiteboard, and noticed that my solution was becoming more and more complicated, involving more and more state variables, each needing a mutex to protect it. I then stopped, and asked to do this on a laptop. I solved it in less than two minutes.

I could probably improve this skill, had I thought it was useful for anything other than job interviews. Had it been the other way around, I would obviously be useless as a programmer. As things stand, however, I think a bare minimum of keyboard and a text editor (any text editor, notepad included) is not much to ask. To date, I have always asked for one, and have always been given one, so it has never been a problem. I don't know how big companies, such as Google, would handle such a request, however.

Shachar

Depends on the encryption method you use

<shameless_plug>rsyncrypto</shameless_plug>

Shachar

We ended up forking - mawstats.

Deliberations over whether to fork jawstats was a hard one. The extension was part of a project done for a client of ours. We ended up deciding that we did everything we could to contact upstream, and it was either fork or keep things to ourselves. Luckily, the client (who is the one paying the actual money :-) agreed, and this is the result.

If you have no choice, then you have no choice.

Shachar

From http://rsyncrypto.lingnu.com/index.php/Algorithm: "The entire rsyncrypto can be summarized with one sentence. Rsyncrypto uses the standard CBC encryption mode, except every time the decision function triggers, it uses the original IV for the file instead of the previous cypher block."

So you're basically dividing each file into chunks and encrypting them separately using the standard algorithm. Seems pretty safe to me. The only obvious leakage is that an attacker can tell if two files are substantially identical. In this, it is similar to ZIP files whose members are encrypted.

The decision function is that used to periodically reset gzip: the sum of the previous 8196 bytes is evenly divisible by 4096. Personally, I'd have used the same Adler-32 algorithm that rsync uses internally.

It appears that there are options for rsync encrypting files on the far end. rsyncrypto might be just one of these. I have not used them but I remember them coming across my 'radar screen' in the past.

Rsyncrypto

http://rsyncrypto.lingnu.com/

Encrypts while making the above sentence untrue - a small change in the file results in a (relatively) small change in the cypher text.

It's stable, been around a while, and is written by me :-)

Shachar

P.s.
It is, in fact, written by me for the purpose of a commercial backup service. The software itself, however, is free.

Have you checked out rsyncrypto?

For businesses that make a living from selling support (SugarCRM, RedHat etc.), the path is a different one.

First, you create the project. You keep updating it and improving it, until it forms a community. You keep mentioning that you also offer commercial support for the project, but until it has a community of early adopters, no one will pay you to support it.

If you manage to cross that sea, however, there is good money in FOSS. RedHat make all their money by selling support for the product after they managed to turn it into a standard. MySQL argueably do the same (they also try to sell licenses, which is something I'm not sure I agree with). SugarCRM are doing the same, though they did annoy the "community" enough to create a split. It'll be interesting to see what happens with that.

The thing to understand here is that you have a very long road ahead of you yet, before you can actually quit your day job for this.

Personally, I moved into the "sell services, base them on FOSS" business. Some of the FOSS involved was written by us, but we never sell the actual software, always the service behind it.

Shachar

Ehhm. I'm afraid that RAID is no substitution for backup. It only protects against ONE of the problems that backup protects against (again, full details at http://lingnu.com/backup.html - my company).

Trying to analyze what you have described, you have a serious problem if old backup tapes turn out to be bad (as they tend to do, over time). You cannot write data to a tape, and then just put it in a safe and expect it to stay there three years later. Every so often, a full (non-incremental) is necessary.

I do believe that the technology we employ for online backups would handle your case fairly well, though. We use rsync-friendly encryption (we developed it, but we open sourced the actual technology - http://sf.net/projects/rsyncrypto). This means that you don't have to upload the entire 700GB the whole of the time. In fact, for all practical purposes, you only upload the data that you have changed.

Still, with this magnitude of data, I'm not sure that online backup is the right path for you. If you wanted to back up the entire 700GB with our service, I may be able to get you a price as low as ~5$/compressed GB/month. Assuming we believe the industry that compression is 1:2 ratio, that means you need to pay for 350GB, or 1750$/month. Most companies prefer to roll their own at this price point.

Shachar

No, it does not.

http://www.lingnu.com/backup.html

You hack one server. One copy of the data gets corrupted. Second copy, however, is on a server that can only initiate outgoing connections. You cannot hack that one from outside. By the time the data gets synced, the hash proves to be wrong, and we know we were hacked. Restore from good backup, and we're done.

Shachar