Slashdot Mirror

← Back to Stories (view on slashdot.org)

Bittorrent Implements Cache Discovery Protocol

Posted by ryuzaki0 on from the middlemen-i-wouldn't-want-to-be dept.

An anonymous reader writes "CacheLogic and BitTorrent introduce an open-source Cache Discovery Protocol (CDP) that allows ISP's to cache and seed Bittorrent traffic. Currently, Bittorrent traffic is suffering from bandwidth throttling ISP's that claim that Bittorrent traffic is cluttering their pipes. This motivated the developers of the most popular Bittorrent clients implement protocol encryption to protect bittorrent users from being slowed down by their ISP's. However, Bram Cohen, the founder of Bittorrent doubted that encryption was the solution, and found (together with CacheLogic) a more ISP friendly alternative."

45 of 170 comments (clear)

  1. i wanna go fast by MrSquirrel · · Score: 5, Funny

    We have the technology -- we can make him stronger, faster, better! ...now, if only there were some more seeders.

    --
    A computer once beat me at chess, but it was no match for me at kick boxing.
    1. Re:i wanna go fast by timeOday · · Score: 3, Insightful

      Wouldn't this technology make your ISP a seeder? Now that would be fast.

    2. Re:i wanna go fast by arivanov · · Score: 4, Interesting

      More likely fast in terms of "lawyers homing fast".

      Anyway, the problem is elsewhere. It all boils down to Telco thinking combined with incompetence. ISPs have degenerated to the point of being either telco resellers or telco wannabies and they are no longer capable of solving a trivial problem through network design and product definition. So they try a silver bullet (CacheLogic) or a big stick (fare share, bandwidth throttle and "kick the hogs" policies) instead.

      Once upon a time around 10+ years ago it was commonplace to charge people for traffic and to have multiple charge categories with local traffic free or nearly free. That was in the days before the big telcos became interested in the Internet. When the big telcos became interested in the Internet the first thing they pushed for was to increase port density and bandwidth on access concentrators and routers. In order to do this the vendors killed the bandwidth accounting features. Best example - Cisco Netflow stopped working in 1999-2000 with the release of CEF (can give plenty of other examples actually).

      As a result of the normal equipment upgrade cycle 10 years later there are very few devices out there capable (and tested in real deployments) of bandwidth accounting on the edge. Even if there were, as a result of the "people upgrade cycle" there are even less people in ISP business development and engineering capable of defining, developing and rolling out a bandwidth accounting based product.

      If the charging was based on bandwidth accounting and local traffic was free (or seriously discounted) the "bandwidth hogs" problem would go away right away. So will most of the "Joe Idiot" problems related to people not cleaning their zombie machines (when these start costing them money they will be cleaned right away). People will again start running local network services for community purposes. For example I used to run centralised network backup for some friends but I stopped as eats the monthly "fair use" quota allocated to me by the ISP in less than a week. And so on.

      The only people who will actually suffer from the reintroduction of bandwidth and differentiated charging will be c***sucking freeloaders of the Nichlaus Zenstrom "it is my right to steal your bandwidth for my service" variety. And CacheLogic (the economical drive to buy their device will go away). Frankly, good bye and good riddance.

      --
      Baker's Law: Misery no longer loves company. Nowadays it insists on it
      http://www.sigsegv.cx/
  2. Off the cuff thought by Arimus · · Score: 5, Interesting

    Just read this and wonder what the legal position for ISP's will be with regards to caching non-legal P2P files (warez, music files etc)?

    With the files being on my PC and served from my PC I'm the responsible party... if the ISP then is caching that data to make it more available (speed/latency/load reduction etc) then the ISP could be deemed to being a party to an illegal act...

    --
    --- Users are like bacteria -> Each one causing a thousand tiny crises until the host finally gives up and dies.
    1. Re:Off the cuff thought by zhouray · · Score: 5, Informative

      I assumed you didn't read the article. It says "only for commercially licensed content".

    2. Re:Off the cuff thought by muftak · · Score: 5, Insightful

      On the cache the files are stored as file chunks, with only a reference to the file hash value, not the filename. So the ISP has no idea what is in the cache, so it is the same as the file being passed through their network.

    3. Re:Off the cuff thought by Andy+Dodd · · Score: 5, Interesting

      It looks like (from TFA), there will be restrictions in place that only allow caching of non-copyrighted, legal content.

      It goes a LONG way towards legitimizing BitTorrent in case anyone tries to sue Bram, but contains no real-world benefits.

      If ISPs want to reduce bandwidth overuse by seeders... Just IMPLEMENT MULTICAST ALREADY!

      Yes, I realize multicast has historically presented major problems in scalability at the backbone router level, but with modern processing power and memory economics, it shouldn't be that difficult to implement now, and in the end presents far more benefits (massive reduction in bandwidth usage) than its disadvantages (backbone routers need some pretty hefty amounts of memory to track all of the multicast groups.)

      Even "limited" multicast solutions like xcast (explicit multicast - basically instead of sending to a "multicast group" an IP datagram is given multiple destinations) would result in MASSIVE reductions in bandwidth usage by P2P applications like BitTorrent.

      Due to the nature of BitTorrent and how it is used in general, caching is just an extremely hackish and limited way of implementing a shitty form of multicast... If the backbone supported multicast, there wouldn't be any need for caching of torrents.

      --
      retrorocket.o not found, launch anyway?
    4. Re:Off the cuff thought by Sark666 · · Score: 5, Interesting

      When bittorrent 4.2 was released, there was already mention of this, and I thought ya right the isps will help with torrents, but supposedly isp caching (even copyright material) is allowed under the dmca.

      http://www.slyck.com/news.php?story=1231

      http://www4.law.cornell.edu/uscode/html/uscode17/u sc_sec_17_00000512----000-.html

      " If a file shows up on the network frequently, the cache stores that file so that its seeded in the network rather than by peers. ISPs appreciate this because their access networks are terribly congested with P2P traffic. Caches are legal and covered explicitly in the DMCA"

    5. Re:Off the cuff thought by mzs · · Score: 5, Insightful

      And who doles-out the multicast group addresses? I think the problem is harder than you think at first glance.

    6. Re:Off the cuff thought by xenocide2 · · Score: 2, Informative

      The thing is, the intention of the law for caching is for otherwise legal copies. As in, graphic images for popular websites like slashdot are allowed to be cached, so long as you obey industry standard refresh requirements. And section E of the conditions pretty much makes it clear that observing copyright is more important than saving bandwidth.

      Which is to say, because the internet is incredibly efficient at duplicating binary information, rather than mere transferral, machines involved in improving this process aren't held as infringing on copyright by virtue of simply doing their job storing packets after they've reached their destination. This is similar in intent to the laws that say you're not infringing for having a copy of a program on disk and in memory, or in residual backing store.

      Moreover, the cache requires that both of you request a specific material from an identical person. It remains to be seen if a chunk, the small parts of a file that you distribute among peers, qualifies as a material. And even if it did, there's the problem that you likely haven't requested the chunk from the same person. The law simply wasn't written with bittorrent / swarming style p2p in mind, and a literal interpretation would likely fall flat in court.

      At any rate, if an ISP choose to seed, say a movie, that would likely cause a ruckus with the owner. I'm no lawyer but it seems plausible that such an action would violate 512 (b) 1 A, which requires someone besides the ISP to offer the data. In otherwords, the ISP can't be the source of copyright violation and get away with it. Not to mention consumer ISPs would rather sell you the movie with their media partners rather than sell you bandwidth and piss those partners off. The short and long of it is that if you're gonna cache bittorrent, you might as well just use something like newsgroups instead.

      --
      I Browse at +4 Flamebait

      Open Source Sysadmin

    7. Re:Off the cuff thought by Pxtl · · Score: 2, Interesting

      I think you've hit the nail on the head with "ya right".

      Doesn't matter if ISP-side torrent-cacheing woudl turn every computer into a supercomputer - ISPs won't do it, for a variety of reasons:

      1) Legal liability, obviously. Sure, it's probably fine, but not-caching torrents is definitely fine, which is better than probably fine. This is called the "chilling effect".

      2) Easier just to not do it. The torrent-cache is one more system to maintain that they'd probably just rather do without. For any software problem there are two solutions, the right solution and the easy solution - and an ISP will always choose the easy solution unless it offends 99% of their customers.

      3) The only people who want this feature are the kind of users the ISP would rather be rid of. You know, the users that actually use their service instead of just checking email once in a while.

      3)

    8. Re:Off the cuff thought by Spezzer · · Score: 4, Insightful

      Some people in networking research believe that the problem with Multicast (and even QoS) has nothing to do with scalability, but more with economics. Although in this case, ISPs would reduce traffic going through their network by enabling multicast, there is no popular method of accounting for internal traffic when multicast is enabled on all routers. For most ISPs this is unacceptable, since large customers are billed based on the amount of traffic sent. Since there's no economic model developed for multicast-traffic, ISPs would rather throttle back BitTorrent than enable multicast. Someone please correct me if I'm mistaken on any of these points.

      Most networking researchers seem to believe multicast is technologically feasible and helpful, which is why a lot of Internet architecture research seems to provide methods for multicast, even though hardly anybody uses it today.

    9. Re:Off the cuff thought by silas_moeckel · · Score: 3, Informative

      They are allready allocated. Modern multicast uses a source IP / port, multicast destination address /port tuple(sp?) so realy you can pick any of the piles of multicast addresses to use traffic is split up based upon the tuple that you joined. Lower end gear hasent been as specific as higher end gear in splitting up traffic leasing the OS to remove anything unwanted but modern switches listen in on multicast setup to be more specific but those times are going away as the old gear gets aged out (managed 100bt gear is about the newest stuff that would do this)

      --
      No sir I dont like it.
    10. Re:Off the cuff thought by nuintari · · Score: 2, Informative

      I work for an ISP, and no, you are incorrect. ISP's are not Telco's and are therefore, not covered by common carrier status. You share illegal files, your ISP is just as liable as you are. If the copyright holder files a complaint with the ISP, and the ISP doesn't deal with the issue to the holder's satisfaction, the ISP can be sued as if they were directly responsable.

      Doesn't mean it happens, any smart ISP noc shuts anything down as soon as they get a complaint. Frequent offenders might just find themselves being contacted by the copyright holder directly.

      As for server's at my core network caching bit torrent, and sending it out all over the place, no thank you. Bandwidth costs money people. I'd rather the customer saturate the hell out of his own connection, encourage to throttle it back a little, than have 10 mbit of zero revenue generating bandwidth flying out over my upstreams.

      Oh, and that's CISCO Discovery Protocol, get your own acronym.

      --

      --Nuintari

      slashdot : where an opinion can be wrong.

    11. Re:Off the cuff thought by aprilsound · · Score: 3, Interesting
      You choose one at random. The chance of a collision is low, and if it is detected, you randomly choose again. Not a big deal.

      In response to the GP, it's not even a matter of implementing multicast. Almost all of the networking hardware out there has it in place, it's just turned off.

      The reason? The original implementation is hard for ISPs to charge for. But there is hope. At SIGCOMM 2006, there was a proposal that would be more ISP friendly, with a minimal performance hit. Its called Free Riding Multicast and essentially piggybacks off BGP's unicast routes.

    12. Re:Off the cuff thought by matts-reign · · Score: 3, Informative

      I'll give you an example how it would be used in a bittorrent style network application:

      I am peer 1. I have section 4 of "the file". In current bittorrent, I upload this file to peer 2. However, peers 3, 7, 24, 23, and 15 need that chunk too. With multicast, I can send the file to all of them at once.

      Sure, it has to be at the same time. There may be times when a portion of a file is sent to only 1 user. But with significantly large peer swarms, it is useful.

      --
      Waffles rock.
    13. Re:Off the cuff thought by markom · · Score: 2, Informative

      Well, there is only one problem with this. Multicast in itself is connectionless and doesn't work with TCP. If I'm not much mistaken, bittorrent is TCP. To make it work with UDP, the whole new mechanism would have to be developed for it to be reliable. There are solutions like "reliable multicast" that has fallback to unicast, but on a large scale, this won't work. Benefits of multicast would be absolutely minimal.

  3. Possible legal problems by woodhouse · · Score: 4, Insightful

    Given that a lot of torrents are copyrighted content, are ISPs really going to want to do this? The moment they start caching these files on their servers, they become a huge target for lawsuits.

    1. Re:Possible legal problems by MrZaius · · Score: 2, Interesting

      Given that a lot of torrents are copyrighted content, are ISPs really going to want to do this? The moment they start caching these files on their servers, they become a huge target for lawsuits.

      On top of that, what torrents are ever so common as to warrant the use of a cache? There are certainly legitimate users of bittorrent, if you can limit the cache to legitimate content. But what torrents would ever be accessed so frequently by individual users on any given network that this would make sense? My employer's just ~300-400 customers strong, but I don't see how this could be useful to any ISP, given that the largest would probably only benefit if the caches were replicated and stored close to the users.

    2. Re:Possible legal problems by cmeans · · Score: 2, Informative
      From the article:

      ...downloads will be accelerated instead of throttled. However, only for commercially licensed content.

      --
      Give a hand, not a hand-out.
    3. Re:Possible legal problems by Bogtha · · Score: 4, Informative

      Given that a lot of torrents are copyrighted content, are ISPs really going to want to do this? The moment they start caching these files on their servers, they become a huge target for lawsuits.

      They already do it with HTTP proxies and Usenet servers without getting sued. So long as they are simply complying with a content-neutral communications protocol - which is basically the whole point of an ISP, I don't see how they could be held accountable. Their business is to transport bits in a particular fashion. It's not up to them to decide which bits are "good" bits and which bits are "naughty" bits.

      --
      Bogtha Bogtha Bogtha
    4. Re:Possible legal problems by ajs · · Score: 3, Interesting

      First off, many torrents are copyrighted, but many more are not, and they're both a problem for ISPs, so yes they'll WANT to. The question is CAN they? I thik they can, but have to look over the details more.

      If the system simply facilitiates the protocol blindly, then I don't see how they could be any more to blame for copyright violations than AOL's Web proxies. Sure, gigabytes of copyright violations move through AOL's proxies every day (and get cached to speed up downloads), but they literally don't have the processing power to try to make a distinction. Same goes for the ISPs and BitTorrent (or Gnutella, or any of the other high-bandwidth swarming download technologies).

    5. Re:Possible legal problems by Ant+P. · · Score: 2, Insightful
      On top of that, what torrents are ever so common as to warrant the use of a cache?

      How many people download Linux ISOs using BT? If 30 people on one ISP download a new release, and it's using this, the ISP saves about 20-30GB and the users get the full 300KB/s they pay for instead of 2-3KB/s.
  4. the next logical question... by zonker · · Score: 3, Insightful

    when will this be implemented in azureus and utorrent? i appreciate bram's work immensely but i'm not too keen on his app...

    --

    Large print giveth, and the small print taketh away

  5. Pipes? by norminator · · Score: 4, Funny

    Currently, Bittorrent traffic is suffering from bandwidth throttling ISP's that claim that Bittorrent traffic is cluttering their pipes.

    You mean tubes.

    1. Re:Pipes? by Scorchmon · · Score: 3, Funny

      Not to be confused wtih a big truck. The internet is most definitely not a big truck. You could possibly confuse the two.

    2. Re:Pipes? by MikeWasHere05 · · Score: 2, Funny

      Looks to me like some horses and poker chips could solve the ISPs problem.

    3. Re:Pipes? by TheSpoom · · Score: 2, Funny

      NO! The poker chips will stack up and clog the porn! You meant lottery balls, my friend.

      --
      It's better to vote for what you want and not get it than to vote for what you don't want and get it.
      - E. Debs
    4. Re:Pipes? by x2A · · Score: 2, Funny

      Look, you can't just keep dumping your own private jokes on this slashdot, it can't support them, and results in situations where it can take me 5 days to get the joke.

      --
      The revolution will not be televised... but it will have a page on Wikipedia
  6. Way to pick a new protocol abbreviation by Anonymous Coward · · Score: 2, Insightful

    CDP = Cisco Discovery Protocol
    http://www.javvin.com/protocolCDP.html

    1. Re:Way to pick a new protocol abbreviation by Amouth · · Score: 2, Funny

      ahh just give it spanning tree's abv.. no one cares about it anyways

      --
      '...if only "Jumping to a Conclusion" was an event in the Olympics.'
    2. Re:Way to pick a new protocol abbreviation by ToTheBone · · Score: 2, Funny

      A few alternative suggestions:
      TCP (Torrent Cache Protocol)
      SMB (Storage Method for Bittorrent)
      ATM (Advanced Torrent Method)
      BOOTP (Bittorrent Over Other Temporarystorage Protocol)
      BGP (Bittorrent Gateway Protocol)
      HTTP (Helper Torrent Transfer Protocol)
      NTP (Networkfriendly Torrent Protocol)
      TDMA (Torrent Data Management Advanced)
      TFTP (Torrent File Transfer Protocol)

      I'm sure someone will have a few even better suggestions

  7. obligatory by cli_rules! · · Score: 3, Funny

    Isn't torrents clogging up the tubes the real problem?

  8. Between the... by Known+Nutter · · Score: 2, Funny

    between the clogged tubes and the friggin SNAKES... on a PLANE! I'm not sure what to do...

    --
    Beware of the Leopard.
  9. No, ISP's won't get in trouble. by saleenS281 · · Score: 4, Insightful

    It's no different than them hosting usenet servers. When contacted by copyright holders they are required to remove the infringing material(s). As long as they aren't actively monitoring what they're caching, they aren't required by law to do anything about it. +1 for legal precedence before lobbyists took over our government (at least the telecom portion).

  10. Technical errata by ElephanTS · · Score: 2, Funny

    Currently, Bittorrent traffic is suffering from bandwidth throttling ISP's that claim that Bittorrent traffic is cluttering their pipes.

    Jeez, who writes this stuff? Must be clueless because everyone knows the internet uses tubes. Sheesh.

    --
    spoonerize "magic trackpad"
  11. Locality awareness in the protocol is the answer by sdpinpdx · · Score: 3, Interesting

    No ISP cooperation necessary. This has been tested experimentally a couple of times.

    See http://del.icio.us/tag/p2p+locality

  12. Another Cache? by OverlordQ · · Score: 2, Interesting

    Azureus has supported JPC (http://www.joltid.com/index.php/peercache) for quite a while now.

    --
    Your hair look like poop, Bob! - Wanker.
  13. Why not IP Multicast? by doshell · · Score: 2, Interesting

    Wouldn't IP Multicast be a more appropriate solution to this problem (and, for that matter, also for the whole lot of streaming content that flows on the 'net nowadays)? AFAIK it has been standardised for some time now, both for IPv4 for IPv6. Why, then, is it that multicast is virtually unused outside private networks?

    --
    Score: i, Imaginary
    1. Re:Why not IP Multicast? by Wesley+Felter · · Score: 2, Informative

      IP multicast creates a routing table entry for each group in every router that the group's packets flow through. If Internet users were allowed to create multicast groups, routers everywhere would run out of memory immediately.

      Also, ISPs claim that they don't know how to bill for multicast.

    2. Re:Why not IP Multicast? by modeless · · Score: 2, Informative

      What about XCast? Seems perfect for groups around the size of a typical torrent, and if the torrent gets too large you can just use multiple XCast groups because the number of groups is unrestricted. Even if you need many groups you'll still save a ton of bandwidth compared to unicast.

      Seems to me like the multicast people have been going about it the wrong way all these years, with tons of state inside the network. What happened to the dumb network philosophy? A stateless protocol like XCast is what is needed. I don't know if it can help with the billing problem, but surely the fact that each packet lists all of its destinations can't hurt.

      --
      Firebug. It will make your jaw hit the floor.
  14. Why don't they use multicasting? by mi · · Score: 4, Insightful

    The sender can multicast the file in a loop. The recipients will get the pieces starting from whenever they started "listening" on the ongoing multicast, and then get the earlier parts, when the sender finishes and starts over again.

    This is far more efficient, than for the sender to push the same data to each client in parallel.

    --
    In Soviet Washington the swamp drains you.
  15. JPC by eddy · · Score: 2, Interesting

    Azureus already have LAN Peer Finder and JPC (Joltid Peer Cache). Not sure how this is different from JPC on the practical level:

    Joltid PeerCache (JPC) is a device employed by ISPs to reduce the huge external network bandwidth required to support today's P2P networks. It basically acts as a caching web proxy (like Squid), only for P2P file data.

    Looks like by going its own way, the official client will once again create segmentation, just like with DHT.

    --
    Belief is the currency of delusion.
  16. Re:Locality awareness in the protocol is the answe by Wesley+Felter · · Score: 2, Insightful

    A locality-aware swarming protocol can only discover other peers at the same ISP that are running at the same time, but a cache hosted by the ISP is always running and can serve content that was downloaded by another client earlier (sort of cooperative prefetching). Also, the bandwidth between the cache and a customer is usually going to be much higher than the bandwidth between two customers because of asymmetric connections.

  17. Re:Only for commercially licensed content by rikkards · · Score: 3, Insightful
    Most of the GNU/Linux distributions I've downloaded were via Torrents.


    It's not GNU/Linux distributions that have caused ISPs to decide to bandwidth throttle bittorrents.