A Better FTP?

cppgodjavademigod asks: "I used to work for a company that sold a file transfer product for datacenters. It supported checkpoint/restart, encyrpted password transmition, asynchonous job procesing, etc. Is there an Open Source project that aims to provide a better FTP? I'm looking for something that makes use of multiple paths (for machines connected via more than one network), job restart, job control, secure transmission (over internet), maybe even tunneling over HTTP and redundant servers (via some kind of private P2P protocol)."

Comments

rsync efficient secure file transfers

[Wills]

The rsync algorithm meets most of your requirements. rsync was proposed in 1998 by Andrew Tridgell for efficient secure file transfers. The main points are:

• For efficiency rsync skips any previously received parts of files, a process based on transmitting small checksums instead of large file chunks.
• For security you can tunnel rsync over any secure protocol such as ssh/openssh. If you don't want or need protocol-level security you can tunnel it over http.

The detailed description is here (http://samba.anu.edu.au/rsync/tech_report/), and open-source software is here (http://samba.anu.edu.au/rsync/download.html).

Overall rsync is often much (10x) faster than using compressed file transfers. It is most useful for users who frequently download new versions of packages with significant similarities between successive versions.

Re:rsync efficient secure file transfers

[gopherdata]

rsync is a great for transferring entire directory structures. I use it to keep our development, backup, and webservers up to date. We have about 4 gigs of data on our sites because rsync only transfers the files that have been updated syncing a day's updates across our entire network usually takes only a few seconds. However, as useful as rsync is, it is not a replacement for ftp.