Slashdot Mirror
Faster Feeds Using FeedTree Peer-To-Peer
dsandler writes "Researchers at Rice University have just released version
0.7 of FeedTree, a peer-to-peer
system for distributing Web feeds faster. Instead of polling feeds
independently, FeedTree users cooperate to share news updates
using multicast in Pastry, a scalable p2p
overlay network. FeedTree reduces the update delay for existing RSS and Atom
feeds to a few minutes without putting extra stress on the webserver (anyone
who's ever been temporarily banned by Slashdot's RSS feed knows this is a real
concern). Feed publishers can also choose to push digitally signed updates
for immediate, tamper-proof delivery to subscribers. The client software (download) runs on Linux, OS
X, and Windows, and works with any desktop feed reader."
Now how do we get a plugin for something like this mass installed into browsers like that of flash, so that a web app like google reader (but independently developed, maybe specialized to other things) can take advantage of this idea?
That would make for a real nice way of creating awesome web based feed aggregation tools...
http://www.babysmasher.com
http://www.openingbands.com
It's not "just another p2p", it's a p2p specifically for distributing newsfeeds. Bittorrent doesn't really work to well for that because it doesn't have the infrastructure and downloading the real feed would be easier than downloading the torrent first. This software bypasses any user interaction and grabs it off its established network.
Why is it that when you believe something it's an opinion, but when I believe something it's a manifesto?
It looks like they just re-invented the netnews protocol, which works in a very similar way.
As already mentioned, this doesn't compete with Bittorrent, because bittorrent isn't designed for RSS feeds. Along with the file size issue idonthack mentioned (torrents are only a win when the size of the file being transferred is much much larger than the coordination overhead, generally not the case for RSS), BitTorrent is also not designed for files to change over time; it would require a complete overhaul of the protocol because the file hashes that are the foundation of the protocol would be constantly changing.
There is room for coordination with bittorrent, though; imagine a Pastry-based P2P feed that then used RSS enclosures to tie into a (trackerless?) BitTorrent feed for a fully distributed pod-/vid-/file-casting solution that anybody could run with no fear of the bandwidth involved.
Tack in some sort of P2P web system, and in theory, you could run a massively popular podcast/blog with millions of hits a day off of your cable modem. (Although something with a bit more upstreaming oomph would be good for the rarely-requested content that falls out of the P2P; anyhow, any ol' webhost could handle this kind of bandwidth.)
I think this is a worthy goal, as if nothing else, popular websites run for fun would no longer be faced with the dilemma of advertising to cover bandwidth costs or going offline.
I remember seeing something like this in my logs over a year ago. I would see lines like this in my access log:
/rdf10_xml HTTP/1.1" 200 5322 "" "Shrook/76p (Distributed; +http://www.fondantfancies.com/shrook/distfaq.php) "
66.177.198.139 - Anonymous [04/Apr/2005:03:04:17 -0500] "GET
I haven't seen a hit from this in a while, perhaps that effort didn't gain much traction. Who knows if this one will... I never saw Shrook mentioned on Slashdot.
I wonder: If GMail were to incorporate an RSS reader (the way Thunderbird does), it could potentially update many, many users with a single hit of each RSS site.
I'm leaning towards using RSS as a way to do announcements rather than maintain a mailing list. Rather than tell me you want me to send you updates (and deal with being potentially a spammer, deal with your unsubscribe, your email address change, etc.), just poll my site every so often (days, for the lists I'm talking about; hours, for Slashdot) and let it show up in your mail queue.
The idea isn't quite ready for prime time; too few people use RSS. But GMail could make that happen in one fell swoop. Well, two fell swoops: you'd need some sort of browser extension to make the little orange "RSS feed" button notify GMail.
I wonder if just having GMail (and hotmail, aol, and yahoo) handle that would solve the problem to the point where we no longer needed a P2P RSS distribution system.
Alternatively, if ISPs were to cache the RSS feeds the way some do with certain web pages, that might also take a lot of the load off. People will still impolitely set their RSS readers to check the feed every 10 seconds, but at least it never gets out onto the backbone if it's cached at the ISP.
Try saying that headline 5 times fast!
You never expect irony, do you?
Want to be a professional wrestler? Visit www.iyfwrestling.com
@iyfwrestling
As a Rice Computer Science student I would like to point out that Pastry actually originated at Rice, under Dan Sandler. The first framework was in Java. You can see from his web page that he's responsible for FeedTree, too.
Microsoft Research became interested in the product and ported it to C#, effectively turning it into the form it is now. Many classes at Rice have now "backported" it, I guess you could say, and it's used for many of our classes that involve distributed networks, such as the current COMP 410 class which has previously turned out distributed file and process system codename Voltron.
Here's a link to the paper co-authored by Sandler and others at Rice.