Faster Feeds Using FeedTree Peer-To-Peer

dsandler writes "Researchers at Rice University have just released version 0.7 of FeedTree, a peer-to-peer system for distributing Web feeds faster. Instead of polling feeds independently, FeedTree users cooperate to share news updates using multicast in Pastry, a scalable p2p overlay network. FeedTree reduces the update delay for existing RSS and Atom feeds to a few minutes without putting extra stress on the webserver (anyone who's ever been temporarily banned by Slashdot's RSS feed knows this is a real concern). Feed publishers can also choose to push digitally signed updates for immediate, tamper-proof delivery to subscribers. The client software (download) runs on Linux, OS X, and Windows, and works with any desktop feed reader."

Comments

Re:Why?

[idonthack]

It's not "just another p2p", it's a p2p specifically for distributing newsfeeds. Bittorrent doesn't really work to well for that because it doesn't have the infrastructure and downloading the real feed would be easier than downloading the torrent first. This software bypasses any user interaction and grabs it off its established network.

Already getting hit by Shrook

[Refried Beans]

I remember seeing something like this in my logs over a year ago. I would see lines like this in my access log:

66.177.198.139 - Anonymous [04/Apr/2005:03:04:17 -0500] "GET /rdf10_xml HTTP/1.1" 200 5322 "" "Shrook/76p (Distributed; +http://www.fondantfancies.com/shrook/distfaq.php) "

I haven't seen a hit from this in a while, perhaps that effort didn't gain much traction. Who knows if this one will... I never saw Shrook mentioned on Slashdot.

Rice made Pastry, too.

[xiphoris]

As a Rice Computer Science student I would like to point out that Pastry actually originated at Rice, under Dan Sandler. The first framework was in Java. You can see from his web page that he's responsible for FeedTree, too.

Microsoft Research became interested in the product and ported it to C#, effectively turning it into the form it is now. Many classes at Rice have now "backported" it, I guess you could say, and it's used for many of our classes that involve distributed networks, such as the current COMP 410 class which has previously turned out distributed file and process system codename Voltron.

Here's a link to the paper co-authored by Sandler and others at Rice.

Re:Rice made Pastry, too.

[dsandler]

Actually, my advisor, Peter Druschel, developed Pastry with Ant Rowstron (of Microsoft Research). Since then, a number of bright researchers from Rice and elsewhere have contributed to the project; their names and publications are listed on the official Pastry website.

There are a number of implementations of the Pastry design; FeedTree uses the Java-based FreePastry package, which is under active development by Rice and the Max Planck Institute for Software Systems and is available under a BSD-like license. Other interpretations include MS Pastry (C#, used in COMP 410 as you point out) and the Bamboo DHT (Java, inspired by Pastry and developed at UC Berkeley).

Re:GMail RSS

[thing12]

Gmail has Web Clips, that's almost to what you want. But what you really want is for Google Reader to be integrated into Gmail. It probably won't be too long before that happens anyway.

Re:Feed Reader

[Anonymous Coward]

NewsFire has a slick, uncluttered interface, and can be configured to poll FeedTree very frequently. NetNewsWire has been around a bit longer and is quite popular, but its minimum refresh interval is 30 min.

Re:Disturbing trend: MS Funding kills Java App for

[xiphoris]

I'm not entirely sure what happened at Rice w.r.t. Pastry. What I was told by Dr. Wong that Rice had a Pastry version, MS adopted it and converted it to C#, then allowed us to use it freely. All of this was part of an elective class called COMP 410 that students take. Basically, a team of 10-20 people act like a software company, self-organize, meet with a "client" (professor acting like one) and build a huge system.

And yes, we use entirely Microsoft software. But I think it's a good thing. When I took it, Microsoft gave us copies of Visual Studio 2003, SQL Server, and funding for some tablet PCs to use as part of the project. I thought it was a *superb* experience to work with so much real-world technology.

Yes, I suppose one could say that MS is stifling open-source competition... but seriously, we were building an application that used a distributed cluster of SQL Server databases, transactionally changed by Enterprise Services features with Event Queueing; all of this also used a distributed file and processing system based on Pastry (C#). Getting all that to work together with open-source in a single semester would be quite a challenge. What database would we even use? MySQL is definitely not capable of that. And Oracle isn't free.

So, in this case at least, I think Microsoft's support of us has been positive for students. We are not just a Microsoft shop -- there is even a research group at Rice called the Programming Languages Team, which focuses almost exclusively on Java for research projects. I'm currently involved in improving the open-source, student-oriented Java IDE called Dr Java, which is under the purview of PLT.

*Pant*

Well, I'm sorry this turned into a rant. I guess my point of this: Microsoft has not caused Rice to give up open-source software or anything like that. In reality, their funding has exposed us to more software and more systems than we would have otherwise. I think that is a Good Thing.

It is neither a good thing for students to be exposed solely to OSS, nor solely closed-source industry software. A university should educate well-rounded people, and much like liberal-arts universities require students to take many subjects, Rice exposes CS students to different technologies and environments in its computer science program. Otherwise, how can I ever decide which is best for a task?

[Note: I am heavily, personally in favor of Microsoft software and have accepted an internship with them in the C# Compiler group next summer. But this doesn't mean I dislike Java or OSS; I don't see why there has to be a conflict at all. Use whatever tool suits you best.

But that's just me. I'm going to do *my* best to make C#.NET the best language it can be. If you like Java, fine! We can learn from another :]