Slashdot Mirror


Ask Slashdot: Do You Move Legal Data With Torrents?

An anonymous reader writes "We've recently seen a number of interesting projects come from bittorrent.com, including Sync and SoShare. I sometimes use torrents to move several GB of data, especially when pushing large bundles to multiple destinations. It's mostly a hodgepodge of open source tools, though. Apart from anecdotes and info from bittorrent.com, details are thin on the ground (e.g. the Blizzard Downloader). I have two questions for the Slashdot community. 1) Do you use BitTorrent to move data? If so, how? i.e. What kind of data and what's the implementation? 2) If you've looked at torrent clients/tools, what's missing in the open source ecosystem that would make it more useful for moving around large blobs of data?"

16 of 302 comments (clear)

  1. I use it for linux distributions by Qzukk · · Score: 5, Interesting

    The entire point of swarm topology is to move data to a lot of places at the same time. If you just need to get data from A to B without sharing it with anyone else, rsync it.

    --
    If I have been able to see further than others, it is because I bought a pair of binoculars.
    1. Re:I use it for linux distributions by TooMuchToDo · · Score: 5, Interesting

      While working at Fermilab on the LHC CMS data taking team, I used bittorrent to speed up re-installs of thousands of worker nodes. I was able to saturate 10Gb Ethernet links this way, and could reinstall ~5500 Linux boxes within 10-15 minutes (with only two initial OS source servers).

      Yes, Bittorrent is not just for piracy.

    2. Re:I use it for linux distributions by corychristison · · Score: 4, Interesting

      To be fair, rsync + ssh is equally secure as scp. I actually think rsync uses scp in that situation (please correct me if I am wrong).

      The advantage I see rsync having is it is useful for automated (backups) of a large collection of files vs gzipping and copying via scp.

      Although git also does a great job of that with concurrent revisioning built in.

      It all boils back down to using the right tool for the job.

    3. Re:I use it for linux distributions by Anonymous Coward · · Score: 5, Funny

      I nearly jizzed in my pants reading this. Nerdgasm.

    4. Re:I use it for linux distributions by TheCarp · · Score: 4, Insightful

      $ env |grep RSYNC
      RSYNC_RSH=ssh

      Worth putting right in /etc/profile so anyone who doesn't want it can disable it if they want.
      It is an entirely sane default.

      --
      "I opened my eyes, and everything went dark again"
  2. Absolutely by Anonymous Coward · · Score: 5, Funny

    All of my Torrents are legal data. What else would I use Bittorrent for besides Linux distros and Humble Bundle games?

  3. Yes, I have by EkriirkE · · Score: 5, Interesting

    Many times a person is searching for a program to do something by keyword instead of software title, and for free. Torrent sites are a common place to go for something free. I just generate a .torrent for my software(s) and upload it to a few big trackers and the others seem to pilfer it from there. Just make sure the filenames and titles are relevant. It's like SEO, but TTO: Torrent Tracker Optimization.

    --
    from 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
    to 45 2F 6E 40 3C DF 10 71 4E 41 DF AA 25 7D 31 3F
  4. Move Legal Data With Torrents? by MarkvW · · Score: 5, Funny

    Only a complete fucking moron would move legal data with torrents.

    A lawyer is obligated to preserve his clients' confidences. When you store your information on somebody elses server or servers you are giving up custody and control over some of those confidences. In that situation you are entirely dependent upon the strength of your encryption.

    That encryption might be good today or tomorrow, but how good will it be five years from now or ten years from now when quantum computing or the next best thing becomes available for codebreakers.

    Don't risk a lawsuit from a pissed client!

  5. Can you use it to move a 300 megabyte hosts file? by Anonymous Coward · · Score: 5, Funny

    inquiring minds want to know.

  6. Linux ISO's mostly by dlapine · · Score: 4, Informative

    At work I need to install several different types/versions of linux OS's for testing. I always torrent the ISO as a way of "paying" for the image that I'm using.

    A few years back, we did some experimenting with torrents over the Teragrid 10GBe backbone, to see how well that worked over the long haul between IL and CA. With just 2 endpoints, even on GBe, it wasn't better than a simple rsync. We did some small scale test with less than 10 cluster nodes on one side, but still not as useful as a Wide Area filesystem we were testing against. Bittorrent protocols just aren't optimized for a few nodes with a fat pipe between them.

    I am interested in looking at the new Bitorrent Sync client to see how thanks for our setup. We have many users with 10's of TB's of data to push around on a weekly basis.

    --
    The Internet has no garbage collection
  7. gittorrent by lkcl · · Score: 5, Interesting

    the one thing that would help enormously would be to have git be *truly* peer-to-peer distributed. not "yeah shure mate you can always git pull and git push, that's distributed, and you're a peer, right, so... so... git is peer-to-peer and distributed, so what are you talking about you moron??" but "at the network level, git pull and git push have a URL type that is **TRULY** peer-to-peer distributed. to illustrate what i mean, i would like to be able to do the following - with all that it implies:

    git clone magnet://abcdefg0123456789/gittorrent.git

    if you're familiar with magnet links, you'll know that there is *no* central location: a DHT lookup is used to find the peers.

    now, what wasn't clear to the people on the git mailing list when i last looked at this, was that it is possible to use bittorrent to do git pack objects, by creating a file named after the pack object itself. and what wasn't clear to sam (the last person who tried to put git over bittorrent) was that you *MUST NOT* make use of bittorrent's "multiple file in a torrent" feature, because bittorrent divides up its data into equal-sized blocks that *do not* line up with the files that are in them, which is why when you download one file in a torrent you almost always end up with the end of its preceding file and the start of the one after it, as well.

    the idea i came up with is that you create *multiple* torrents - one per git object (or git pack object). if you want to pull a tree, you create a torrent containing *one file* which is the list of objects in that tree; gittorrent would then know to map each of those objects onto yet *another* torrent (one per object), repeat until all downloading happily. gittorrent objects are of course named after the hash, so you can pretty much guarantee they'll be unique.

    and, adding in a DHT (a la magnet links), you are now no longer critically dependent on something like e.g. github, or in fact any server at all.

    to answer your question in a non-technical way, mr anonymous, i think you can see that i feel it would be much more useful to have development tools that use bittorrent-like protocols to share files-as-revision-controlled-data (and, if you've seen what joey hess is doing with bittorrent you'll know that that's a hell of a lot - including storing home directories in git and doing automatic distributed backups)

    1. Re:gittorrent by Qzukk · · Score: 4

      Intriguing idea, but I tried subscribing to your newsletter and I keep getting the Sept. 24, 1998 edition over and over. The problem with using bittorrent for this is distributing NEW data. If the protocol could cope with a seed appending data to the torrent without having to create a whole new .torrent file, then this could be awesome. As it is, you're just changing the problem from "how do I send out new versions of files when I commit something" to "how do I send out new versions of the .torrent files every time I commit something"

      --
      If I have been able to see further than others, it is because I bought a pair of binoculars.
  8. Move my legal data? by Jeremiah+Cornelius · · Score: 5, Funny

    I let my Lawyers handle that. It's what they're paid for, isn't it?

    --
    "Flyin' in just a sweet place,
    Never been known to fail..."
  9. Facebook Does by terbeaux · · Score: 4, Interesting

    Facebook deploys its 4GB binary to its 500,000 servers using a torrent client that has rack and switch affinity. Each client goes for data chunks that are already locally on a rack or switch that it is connected to. That is a crap-ton of data.

  10. Re:Patent Trolls? by Nocturnal+Deviant · · Score: 4, Funny

    i tend to run through a Romanian based VPN almost exclusively for work(my own personal. I wouldn't trust an outside vps with corporate information) related activity, and i can absolutely confirm what he is saying. I've been on the phone with my vpn provider before(hes an old personal friend), and he says "OH BOY MAIL TIME, now i get to have more firewood from your american lawyers" as to which we both chuckle.

    --
    -Noc
  11. Re:No - Resources by darkain · · Score: 4, Insightful

    Serious question... How long have you been using a network HUB instead of a network SWITCH?