Slashdot Mirror
RSS And BitTorrent, Together At Last
eoyount writes "Wired has an interesting story about a really simple idea I wish I had thought of. Transferring large files across the Internet isn't easy for your average joe, but a combination of RSS and BitTorrent technology might just make it easier - Slashdot ran a previous story on the theoretical blending last year." (LegalTorrents is run by the strangely familiar simoniker, who wrote a short piece on the O'Reilly Network about how it was set up, and offers observations on how well the combination fares.) Update: 03/17 21:45 GMT by T : Ernest Miller submits two related postings he's written on RSS+BitTorrent, a combination he calls "broadcatching."
I've been writing extensively on Corante about RSS + BitTorrent, which I call "Broadcatching" here: Broadcatching Archives See, for example, RSS + BitTorrent Roundup - Broadcatching Isn't MS Active Channels and First Broadcatching App Available! (And Related News).
BitTorrent is basically another p2p service, except it's different (yes, i'm trying to be very specific here)
It allows for people to take advantage of bandwith by downloding bits of a large file from different users hosting a 'torrent.' At the end, all these pieces are put together. Yes, it is pretty good.
The problem with bittorrent is that a lot of users disconnect as soon as their download is finished. Won't this be an even bigger problem with game downloads (specifically multiplayer games) since even if the users knows they should stay connected afterwards, they might not since it would lag their game?
RSS feeds are an easy way to move news from site to site. For example, here is Slashdot's RSS feed
You can find more information here
--
Real-time deal updates, over 400 a day!
Try Google, newbie :-)
Oh heck, I'll save you the trouble. Read hereNo new browser protocol is needed.
The technology is already available at http://freecache.org/ [from the peeps @ archive.org]
I don't why many others have jumped on the bandwagon yet.
RSS is "Really Simple Syndication" and it's best thought of as a spinoff to XML. It's a language under which blog-type news-channels can publish their content using, and then the user can use an RSS client that can group stories together into whatever sequence the user wants to see.
It's also seen as a effective way to replace e-mail mailing lists. Instead of getting your newsletters in your e-mail client, open them up in your RSS client which works on a pull basis rather than a push basis, but can still present the content to the user just like an e-mail program might.
It's very different than Active Desktop... that was just the idea of letting IE browser windows be part of the Windows Desktop level so that users could have a frequently-refreshed mini-page of content on their desktop.
Wow! You're right, with one smallish exception:
Please note that you cannot submit a whole site to FreeCache as in http://freecache.org/http://www.rocklobsters.com/ This will not work as only index.html will be cached. You have to prefix every item that you want to have cached seperately.
Using the last THG article as an example, either the Slashdot story would need to point to each page individually via freecache redirection or Tom's Hardware would need to do it.
Not quite as transparent as incorporating BitTorrent into the browser.
Please provide proof that BitTorrent is 'based' on KaZaA sourcecode. Of course, you can't.
:)
:)
I do know that Bram talked with the authors of Furthur, an open-source JAVA P2P for legal content. A few members of the Furthur dev team also work for a company that once did buisness with Sharman Networks, so if anything, KaZaA source may be based on already GPL-ed software... but don't tell anyone that
Of course, all modern P2P is 'based' on Napster or Gnutella (take your pick), so it's all a mute point anyway
does BitTorrent even work under Linux?
Would you like the GUI client or the command line one?
Yes, it works very nicely under Linux.
"Ignorance more frequently begets confidence than does knowledge"
- Charles Darwin
The previous poster is incorrect. BitTorrent has nothing to do with Kazaa (0 lines of code in common).
BitTorrent is open source (MIT license) and written in Python.
Kazaa is closed source, spyware-ridden dreck that was probably written in C++.
Bittorrent protocol has nothing in common with the protocol used by Kazaa (FastTrack). Even their basic P2P topologies are different.
Also, Kazaa is in trouble not for it's protocol, but for running servers that allow piracy, it's just in Kazaa's case one automatically means the other, since the protocol is closed source. Of course, Bittorrent trackers that host pirated material are also susceptable to such troubles - but this has nothing to do with Bittorrent protocol itself.
Uh, yes...
Here, here, and here.You can already subscribe to SuprNova feeds right here. Half way down.
RSS is completely different from usenet. Usenet is very email centric (each post is akin to an email, and the newsgroup is akin to a mailing list, and this is all served up by a dedicated news daemon), whereas RSS is web centric (you download over http from the site that you're viewing syndication of).
Go read Slashdot's RSS feed if you still don't get it. Basically it's just an XML document that defines story "ITEM"s as having a title, link, description and other fields.
right now we've got a 90's-style ol' skool web interface for distributing our media, teamed up with ol' skool ftp/http mirrors distributing things around the globe.
... we are prime users for free, open, public, easy-to-use media sharing technologies, and if the big-guns aren't using it, we share are happy to!
but, we'll definitely use an rss-fronted bittorrent network, if and when it can actually be smoothly integrated with our existing setup.
ampfea stands for 'a meeting place for electronic artists' and its a community-supported media hosting/community service for a bunch of muso's
check out some of our files sometime. its all home-made music...
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
Suprnova is pretty much the major culprit in associating Bittorrent with illegal file trading. Full movies, warez, etc make up the bulk of their content. It's making things difficult for legit uses of Bittorrent, of there are many. Beware if you download anything off Suprnova, the RIAA and MPAA are watching those torrents and gathering some nice logs of IP addresses and times. Remember that by participating in a Bittorrent you are uploading.
Too many good uses for Bittorrent to let the warez kiddies spoil it for us.
...
One more item, while browsing their discussion forums, I found this apropos exchange from Sept, 2003:
t cam.h tml
I have a number of video clips that are being served via freecache from my site at:
http://holden.customer.netspace.net.au/rocke
The site was slashdotted on Saturday. I would like apologise for the additional burden this placed on freecache and any reduction in service levels that occured as a result.
I would be interested in learning more about how the system coped with this event.
Regards,
Mike
In reply:
Mike,
There is no need to apologize for using FreeCache. Your case is exactly why build this system.
And then:
There were 20,000 hits in 24 hours, and plenty of discussion in the Slashdot forums regarding slow downloads - which is to be expected with such a high demand. I found the Status pages were taking much longer to generate than normal.
So it's better, but not perfect.
This site has been offering RSS feed of links to BitTorrent animes for couple of months. Firefox and RSS reader extension are great for pulling down all BitTorrent links.
This problem is easily addressed with multicasting. All a server need do is send a multicast datagram to notify all RSS syndicators that the RSS document has been updated, at which time the syndicators can fetch the new document.
BT is non-linear as you suggest. The n'th person gets the n'th chunk. This still allows for (randomly-caused) relative scarcity of certain chunks (although they are not the last ones!), and that is the problem you notice up around 98%.
No, seriously, try playing a partially complete BT download of an AVI with a player that doesn't look for the index (mplayer, DivX, etc.). The file is missing random chunks, not the end.
The previous sig has been removed due to
Unfortunately, one assumption at the beginning, that the cascading model is best-case performance for BitTorrent, is completely wrong. It's actually worst-case performance.
A scattered model gives BT as taking O(log(number of people)/(number of chunks) + 1) time for everyone to download the whole file instead of O(sqrt(number people)) as claimed in the article.
-----------
100% pure freak
A million years ago (1998?) Wired published a whole edition on Push as the Next Big Thing. It was the first time I was really aware of them being totally wrong. Or perhaps just a bit ahead of their time.
While I think this is a neater solution, there is another product that does exactly the same thing, allow you to subscribe to channels and received pushed content via incentive compatible (you get faster speeds if you upload more) swarms.
It's called kast.
The way I figure it, with this bittorrent-RSS combination and a slight modification of torrent watching sites like animesuki we will essentially have a fansubbed anime online tivo at our disposal. Actually, you could have probably done that even without RSS, though it does simplify matters. The only limitations are our bandwidth and hard drives. Which actually are pretty limiting these days, especially with p2p being frequently capped.
Hell, you could modify an actual TiVo with broadband for exactly this sort of thing, and it needn't be limited solely to anime either. I'm sure it'll be popular with overseas watchers of American TV as well.
The international media and internet companies need to face facts and realize that Video On Demand is a reality and is already extremely popular - but that the shows people are demanding are not the ones the companies have been providing through their own limited, misfocused, and (most importantly) redundant services. Until we see simultaneous worldwide release of all media (including DVDs released simultaneously with the theatrical release) they will find themselves losing what should have been their easiest sales - those to impatiently eager fans.
---If you can't trust a nerd, who can you trust?
Realistically speaking, the biggest problem with Bittorrent is seeding. I think this is how bittorrent works:
.torrent file generated .torrent file is uploaded to a tracker .torrent from the tracker .torrent file, which causes the the bittorrent client asks the tracker for the machines/locations of the seeds and people downloading the file(s) pointed to by the .torrent
.torrent goes missing, the file is inaccessible. If the tracker goes away, the file is inaccessible.
* a file is seeded, and a
* that
* clients who want to download the file download the
* the user opens the
* the client downloads various chunks of the files from both the seeds and the other downloaders
The more people download a file, the better bittorrent is able to spread the bandwidth.
The downside is that if a file isn't seeded, it's no longer available. If a
Bittorrent's main problem right now, which is a client problem, is its upstream usage can easily swamp a home connection. That's just dumb client design.
Upload limiting works, but limits your download speed. The client develoeprs have to recognize that yes, sharing is nice and leeching is bad, but disrupting the users' connection is a Very Bad Thing.
Think of BitTorrent as an updated version of ftp that utilizes the clients upload bandwidth as well as the servers download bandwidth. That's all it is. (Yes, clients become servers even before the download process is complete.)
If the author of BitTorrent could be sued the authors of ftp and apache could be sued as well.
I hope this helps, your misconception of how BT functions is fairly common. It's not the same type of p2p network as Kazaa at all.