Caching Torrent files in DNS

bodin writes "This is a proof of concept version of BitTorrent where the torrent files are transported over DNS. This will of course bog down BIND servers all over the planet. Everyone should be thankful that the files are not sent over DNS."

Comments

BT Itself?

[AntiPasto]

I know its sort of a chicken / egg thing... but can someone explain exactly why BT itself can't be used to distribute torrents, or to share the bandwidth of tracking?

Re:BT Itself?

[damiam]

I know its sort of a chicken / egg thing... but can someone explain exactly why BT itself can't be used to distribute torrents

BT works by connecting to a tracker, which tells it about other hosts downloading the same file. Without a .torrent file, BT doesn't know what file to get or how to contact the tracker. You could distribute .torrent files over BT, but you'd need another .torrent to tell BT where to go to get the first .torrent. It'd only really be practical if you put together a massive collection of .torrents (like the suprnova database) in a .zip file and then used BT to download that.

This was bound to happen

[color of static]

Some of us have been using DNS to get through some really draconian firewalls for ages. It was just a matter of time before we saw someone distribute files this way. What is unique is that they will be cached. This in and of itself is an amazing idea. DNS is well designed to cache for well defined amounts of time.

The load on large DNS servers can grow quickly. I wouldn't be surprised if we see a set of patches coming out for DNS servers to combat this. the question is can we find a TTL that reduces the abuse and still makes it useful.

Re:This was bound to happen

[color of static]

A good DNS cache should still be useful, just not as much as it used to be. The reason, you will reference sites that you use (your mail server, popular websites) thousands of times, while you will access each block in the download once or twice. The hit count should wash out the noise to some degree. Now sites that you only hit once every few hours are likly to require smarter cache design to not become irrelevant to the cache.

Re:Isn't this very easy to combat?

[semaj]

Just ensure that the DNS checks to see that the new file is actually a legal DNS zone file. I assume these things have a required format?

Why would anyone want to "combat" it?

Also, if you'd bothered to read the article (all one page of it) you'd have seen:

torrentToiZoneentries.py converts a torrent (or any file) to zonefile format (example : 0_1198_56633ab0d90f43c68ed1b47358eccfe7 IN TXT "64383a616e6e6f756e63653433...")

Sounds pretty much like putting it into the required format, doesn't it?

Re:"It would be bad..."

[BW_Nuprin]

It seems to me that sites like suprnova.org and other tracker sites already attempt to serve that purpose, in essence they are manually run Torrent DNS servers that are human readable. Their hardware just can't keep up with the demand, so this guy decided to force every DNS on earth to do it for them. A better solution would be for the tracker sites to switch from a web-based solution to a DNS-type (not actually DNS) tracking server, but then they wouldn't have advertising or any hope of fame or revenue. Right now we know who suprnova.org is, and we know they're leet, and they like that. Actually, come to think of it, they prolly don't like this DNS idea either.

Standing up for fun/useless hacks!

[iamdrscience]

There are already a number of posts on Slashdot talking about how this is a really bad idea. I disagree (with a big but). It would almost certainly be a Bad Idea (TM) if this were enacted and people tried to make it the main way of distributing torrents. However, I don't think the idea was meant to do that, I think it was more just to show that it could be done. It's akin to people who get put on the mainpage of Slashdot for hacking their Apple IIe's so that they have ethernet and then writing a simple graphical operating system for them so they can browse the web. Impractical and largely useless? Probably. Interesting and fun hack? Definitely.

Missing the point are we?

[pomac]

If someone adds TXT records to their dnses and in turn to their dns slaves it's their buissnies, how the hell will this affect all dnses available in the world? You generally ask the "owner dns" but yes, if you ask your "ips's dns" or someone elses dns then they might cache the result.

But how bad is that really? How large is a .torrent file? And wouldn't linux distrobutiors enjoy this?

$large_isp has several users who wants to download $linux_dist. The first user gets the TXT record and is off downloading. And the rest of em uses the cached record (if it is cached) in either case $linux_dist's webserver dosn't suffer as hard and they can always add more slave dnses to handle the load. Perhaps users even starts slave servers for that zone to help the dist.

(Is there really a rule that says "you have to cache and store TXT for $TTL time".)

And whats this with spelling? I mean you totally miss the point and... complain about spelling? is that the end of the world? =) /me is also a typical Swedish geek =)

We have the technology!

[DrZaius]

DNS is a resource locating service. For example, SRV records are a nicer way of autoconfiging your network.

With SRV records, you say service.domain = port at host. You could do a dig for ldap.slashdot.org and findout that the ldap server is on port 389 at directory.slashdot.org.

This is a slight extension of this. I don't know the exact implementation, but you could have a zone file that looks like:

'file being served'.bt.slashdot.org SRV 0 0 PORT 'seed host'

You can have multiple SRV's per resource and load balance between them.

DNS is currently used for stuff like this all over the place. We already have the technology. IXFR means we can transfer just the changes in the zones when there are updates.

Last time I checked, DNS is not over loaded and will scale to handle this. Even it 50% of the internet uses BT over DNS, 100% of the internet uses DNS for email, web and so forth. Every time an email is delivered, there are at least 6-10 DNS queries.

DNS will not be bogged down.

Re:Speling?

[Anonymous Coward]

And i like the way he says he uses Bit Torrent for downloading Linux distributions (or "linuxdistributions", as he puts it), and then links to a site that's known for torrents of a much less legal nature.

I've thought about this...

[cperciva]

I've been wondering about (ab)using DNS for FreeBSD Update -- the idea being that when you're updating a system which is not up to date with security fixes, you might want to be behind a draconian firewall. (The caching benefits of DNS are a non-issue; updating a year-old RELEASE takes only a couple MB.)

In the end, I decided that it would be more trouble than it's worth; but if someone else has written code I can borrow (I haven't looked in detail) then I might reconsider this.

Google cache?

[Trbmxfz]

What about attempting to have Google cache torrent files by using a format that allows them to be embedded in the text of web pages?

Often, slashdotted articles are still available thru Google. That might work.

Re:Ouch!

[ctr2sprt]

I don't know if it's a terrible burden or not, but for very popular torrents (>4000 downloaders, >100 seeds) the trackers will often start choking as they run out of steam. With DNS, though, you have automatic load balancing, so no one server gets hammered as badly.

The basic issue here is twofold. First, how can we get load-sharing for all aspects of BT? And second, how can we do that while keeping BT a "lightweight" program (i.e. one that doesn't require special server programs)? DNS may not be the best answer, but it is an answer.

I don't think it's such a bad idea.

[0x0d0a]

I don't see what the issue is, exactly. DNS data is propagated lazily. The only issues is that you'd have maybe three machines storing the data instead of one.

Unless .torrent files are particularly big -- I happened to have one on my hard drive, which was under 512 bytes.

I'm not sure that there's much point in using DNS to propagate .torrent files -- it seems that USENET or similar would be a better choice, given that they tend to only be useful for a short period of time, that announcements of new torrents is a useful characteristic of a .torrent propagation system, and that archiving torrents is useful -- but I also don't really see the harm in this.

Given the kind of load that nameservers happily handle today when you hit a webpage with a number of entries (especially for those annoying little "badges") (and the nameserver potentially gets twenty or more lookup requests all at once), there can't be a huge processing hit.

There *might* be a storage hit...but suppose there are 10,000 torrent files out there, and each is 1K. That's just 10MB of data, and I doubt anyone is interested in storing all available torrents.

Finally, I suppose that bandwidth might be an issue, but I suspect that given the frequency of DNS lookups and the infrequency of someone needing a new .torrent file, the bandwidth will not be an issue.

I have done plenty of fun things with DNS and run a small DNS server, but I will freely admit that I am not a DNS wizard, and leave it to the folks on NANOG to debate the merits of this.

For my money, though, this is cute and not harmful at all, though it might not be particularly useful.

A Friendlier Approach

[AmVidia HQ]

I've circulated the idea of using DNS around the BitTorrent mail list for some time now, but it uses the DNS in a different way from distributing the torrent files themselves through DNS caches. It's doubtful putting data inside caches would "bog down the DNS system", but to turn DNS into essentially web servers is:
1. An ugly, ugly hack, and a wrong tool for the job (tm)
2. Wrong using others' resources in a way that is not intended (serving binary data)

My goal of using the DNS is however the same: solve the dead trackers problem. But to serve data directly from DNS, my idea to use what DNS is for: route traffic. You request the torrent's 40 byte info_hash as a subdomain of the DNS server's domain, and it returns you the optimal tracker's IP. For example:

{torrent_hash}.bthub.com

and bthub.com will be where i'll host the DNS service. It's basically a dynamic DNS server, updated frequently, of all known torrent hashes from different trackers in its database (my estimation is around 15000 - 20000 torrents, or subdomains. I know because I run a BT search engine, see my sig). This way, you don't "hack" the DNS, but you get the benefit of:
1. Dynamic routing to the tracker with the most ppl downloading your file, or the biggest "swarm", and hence the highest speed
2. Re-routing to a SINGLE new tracker when one dies

The 2nd benefit is important, as the case today is ppl just randomly pick a new tracker when the original dies, without knowing how big the swarm is on the new tracker. This splits the original torrent swarm across multiple trackers, which don't talk with each other (yet).

There are some discussion on the mail list of adding connectivity between the trackers, similar to how you link IRC servers together into a network. But that's no where near a proposal, and using DNS to act as a "hub" for the trackers would need the least effort and changes to the BT protocol and trackers.

I discussed this idea in detail on this thread "Decentralizing trackers: Use hubs", feel free to take a look.