Slashdot Mirror
Ask Slashdot: Do You Move Legal Data With Torrents?
An anonymous reader writes "We've recently seen a number of interesting projects come from bittorrent.com, including Sync and SoShare. I sometimes use torrents to move several GB of data, especially when pushing large bundles to multiple destinations. It's mostly a hodgepodge of open source tools, though. Apart from anecdotes and info from bittorrent.com, details are thin on the ground (e.g. the Blizzard Downloader). I have two questions for the Slashdot community. 1) Do you use BitTorrent to move data? If so, how? i.e. What kind of data and what's the implementation? 2) If you've looked at torrent clients/tools, what's missing in the open source ecosystem that would make it more useful for moving around large blobs of data?"
The entire point of swarm topology is to move data to a lot of places at the same time. If you just need to get data from A to B without sharing it with anyone else, rsync it.
If I have been able to see further than others, it is because I bought a pair of binoculars.
All of my Torrents are legal data. What else would I use Bittorrent for besides Linux distros and Humble Bundle games?
Moving large data requires resources. In the case, of bittorent most things don't qualify because it's a distributed network, if 10 people in the office have the file and all know how t seed / use bittorent, you'd still be throttled by your bandwidth. Bittorent has however time and time again shown that it's distributed architecture can get something out to the masses very effectively.
I mean, other than the Blizzard stuff, no, I don't use bittorrent at all unless I'm downloading movies (usually) or software (sometimes).
Rapidshare/megaupload/etc work much better for my one-off transfer needs, while I leave media distribution to the masses via Youtube, Vimeo, Bandcamp, and media collaboration to Dropbox and sneaker-usbdrive-net (especially for big projects).
If you were me, you'd be good lookin'. - six string samurai
Many times a person is searching for a program to do something by keyword instead of software title, and for free. Torrent sites are a common place to go for something free. I just generate a .torrent for my software(s) and upload it to a few big trackers and the others seem to pilfer it from there. Just make sure the filenames and titles are relevant. It's like SEO, but TTO: Torrent Tracker Optimization.
from 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
to 45 2F 6E 40 3C DF 10 71 4E 41 DF AA 25 7D 31 3F
I specifically do not torrent anything that has copyright issues but I do seed a number of Linux distributions and development tools which do not prevent distribution in their licenses. Downloading anything using torrent is effectively distribution of the material too, so you had better KNOW that the license allows you to make copies and give them away.
You folks that torrent movies and stuff that is not in the public domain are crazy in my book.
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
Only a complete fucking moron would move legal data with torrents.
A lawyer is obligated to preserve his clients' confidences. When you store your information on somebody elses server or servers you are giving up custody and control over some of those confidences. In that situation you are entirely dependent upon the strength of your encryption.
That encryption might be good today or tomorrow, but how good will it be five years from now or ten years from now when quantum computing or the next best thing becomes available for codebreakers.
Don't risk a lawsuit from a pissed client!
inquiring minds want to know.
At work I need to install several different types/versions of linux OS's for testing. I always torrent the ISO as a way of "paying" for the image that I'm using.
A few years back, we did some experimenting with torrents over the Teragrid 10GBe backbone, to see how well that worked over the long haul between IL and CA. With just 2 endpoints, even on GBe, it wasn't better than a simple rsync. We did some small scale test with less than 10 cluster nodes on one side, but still not as useful as a Wide Area filesystem we were testing against. Bittorrent protocols just aren't optimized for a few nodes with a fat pipe between them.
I am interested in looking at the new Bitorrent Sync client to see how thanks for our setup. We have many users with 10's of TB's of data to push around on a weekly basis.
The Internet has no garbage collection
Just about the only time I use torrents is when downloading Linux distributions- Mageia, Fedora, CentOS, etc. Occasionally iso's for grub magic, ultimate boot CD, and such. All of that legal. And I usually leave it up at least long enough that my share ratio is 100% (1.0).
mostly for music that's under creative commons license and the occasional Linux download.
Does this seem like fishing expeditions by patent trolls?
No need to go fishing here.. Just find the torrent tracker and connect... Volia, you have a list of everybody who is distributing the material by IP address. It's all about tracing down the IP's and sending out the collection letters and cashing the checks from there, assuming you actually have the permission of the copyright holder to do so...
"File to fit, pound to insert, paint to match" - Aircraft Maintenance 101
Linux distros, free movies, free games...
I tried to switch to Deluge but it couldn't handle a file with a Japanese character in its name...other than that, only things that I think many torrent clients could use is the ability to accept magnet downloads through a drop folder somehow, and searching & better sorting/filtering options for downloaded torrents.
"When information is power, privacy is freedom" - Jah-Wren Ryel
the one thing that would help enormously would be to have git be *truly* peer-to-peer distributed. not "yeah shure mate you can always git pull and git push, that's distributed, and you're a peer, right, so... so... git is peer-to-peer and distributed, so what are you talking about you moron??" but "at the network level, git pull and git push have a URL type that is **TRULY** peer-to-peer distributed. to illustrate what i mean, i would like to be able to do the following - with all that it implies:
git clone magnet://abcdefg0123456789/gittorrent.git
if you're familiar with magnet links, you'll know that there is *no* central location: a DHT lookup is used to find the peers.
now, what wasn't clear to the people on the git mailing list when i last looked at this, was that it is possible to use bittorrent to do git pack objects, by creating a file named after the pack object itself. and what wasn't clear to sam (the last person who tried to put git over bittorrent) was that you *MUST NOT* make use of bittorrent's "multiple file in a torrent" feature, because bittorrent divides up its data into equal-sized blocks that *do not* line up with the files that are in them, which is why when you download one file in a torrent you almost always end up with the end of its preceding file and the start of the one after it, as well.
the idea i came up with is that you create *multiple* torrents - one per git object (or git pack object). if you want to pull a tree, you create a torrent containing *one file* which is the list of objects in that tree; gittorrent would then know to map each of those objects onto yet *another* torrent (one per object), repeat until all downloading happily. gittorrent objects are of course named after the hash, so you can pretty much guarantee they'll be unique.
and, adding in a DHT (a la magnet links), you are now no longer critically dependent on something like e.g. github, or in fact any server at all.
to answer your question in a non-technical way, mr anonymous, i think you can see that i feel it would be much more useful to have development tools that use bittorrent-like protocols to share files-as-revision-controlled-data (and, if you've seen what joey hess is doing with bittorrent you'll know that that's a hell of a lot - including storing home directories in git and doing automatic distributed backups)
It's a honeypot to trap people that can't tell the difference between copyrights and patents.
Do you even lift?
These aren't the 'roids you're looking for.
Linux ISOs
VM Images
Backup Images
Home Movies
etc...
I got a game called War Thunder via bit torrent. What you do is download a small installer program from the publisher's website and run that. The installer automatically connects to BT seeds and peers and downloads the actual game itself.
There is no other way to get War Thunder. I suppose since they're a small publisher, their web server can't handle distributing the 13 GB game file to tens of thousands of users.
I have a fairly large and growing (3.7TB) dataset that needs to be replicated nightly to a bunch of different sites. By the nature of the dataset nothing is ever removed or changed just new files added. It needs to be copied out to a half dozen locations that have as much outbound bandwidth as the primary. So a cron job sets up the torrent every night and all the remote sites pull data from the primary and reshuffle it between themselves.
No sir I dont like it.
I let my Lawyers handle that. It's what they're paid for, isn't it?
"Flyin' in just a sweet place,
Never been known to fail..."
Bit torrent is a good data -distribution- tool, not a data -mover-, and it would be lousy to play that role. There are at least a dozen possible open solutions for moving data from point to point, but I have no idea why you'd use a protocol/tool stack that are designed for broadcast/graph distribution to do so.
An off the top list:
1. NFS
2. SMB
3. FTP
4. SFTP/SCP/rsync
5. HTTP/HTTPS
6. sz/rz
7. iscsi
8. DFS
9. AFS
10. UFTP/XFTP
The question should really be what exactly do you see ad being deficient about all these protocols that deems it necessary to re-invent the wheel yet again?
Bye!
Punish the technology because of how it is used? I thought we grew past that notion already.
At some point, one of the few remaining ways to get good information and news will be through these rougue channels and methods. Do we have to keep re-hashing the same ridiculous notions? How about we ban types of music based on the fact that thugs and criminals like it and glorify killing?
Facebook deploys its 4GB binary to its 500,000 servers using a torrent client that has rack and switch affinity. Each client goes for data chunks that are already locally on a rack or switch that it is connected to. That is a crap-ton of data.
The big advantage of using BitTorrent over many other protocols for moving large amounts of data (as opposed to distributing it) is reliability - or lack of it. When you're moving large datasets, you don't want it to crap out and not be able to resume 90% in.
Sure - rsync has the ability to resume, but it requires explicit command line options. It's a terrible feeling to realise you just restarted a 10GB+ transfer instead of resuming it.
The folks at Etsy do it to replicate SOLR:
http://codeascraft.etsy.com/2012/01/23/solr-bittorrent-index-replication/
Not sure if that's what you mean.
with just two end points, it wasn't better
For point-to-point transfer to large amounts of data, the protocol does't matter, as long as the protocol is sane. The time spent moving data bytes will be much higher than any protocol overhead. rsync is roughly optimal because it won't transfer portions of the file that the receiver already has. BitTorrent is for distributing data to many destinations.
i tend to run through a Romanian based VPN almost exclusively for work(my own personal. I wouldn't trust an outside vps with corporate information) related activity, and i can absolutely confirm what he is saying. I've been on the phone with my vpn provider before(hes an old personal friend), and he says "OH BOY MAIL TIME, now i get to have more firewood from your american lawyers" as to which we both chuckle.
-Noc
BitTorrent is, conceptually, best used as a reimplementation of multicast. Multicast is probably far more efficient when it comes to the actual data distribution, but multicast (specifically, routing multicast) is one of those blackbox things that not a huge number of people understand. Last time I checked, I couldn't route a multicast source from a Comcast connection and have the data arrive on a receiver on the Cox network.
However, there was still a need for a protocol that could effectively do one to many distribution for the masses. Enter bittorrent. It fills the void nicely, and has the additional benefit that once a receiver has all the data from a given source, it then becomes a source itself, thereby increasing the aggregate bandwidth available for the feed, as well as making the feed resilient.
You know you can limit the number of peers you connect to as well?
E.g. I have it set to a maximum of ten torrents active at any one time. Each torrent can have no more than 60 peers. So, total, 600 peers maximum at any one time. Except, I also have it set so that I do not connect to more than 275 peers, ever. So that 600 number, irrelevant.
I.e. these are all problems that, well, aren't.
HELP MY ACCOUNT HAS BEEN HACKED BY AN ILLIBERAL ART STUDENT SET TO DESTROY THE INTERWEBZ!
...and if the connection breaks or anything happens, you'll have to restart copying everything from the beginning.
Interestingly, rsync can even resume a transfer that was started by scp but then interrupted - neat!