Slashdot Mirror
Is RSS Doomed by Popularity?
Ketchup_blade writes "As RSS is becoming more known to the mainstream users and press, the bandwidth issue reported by many sites (Eweek, CNet, InternetNews) related to feeds is becoming a reality. Stats from sites like Boing Boing are showing a real concern regarding feeds bandwidth usage. Possible solutions to this problem are emerging slowly, like RSScache (feed caching proxy) and KnowNow (even-driven syndication). RSScache seems to offer a realistic solution to the problem, but can this be enough to help RSS as it reaches an even bigger user base in the upcoming year?"
Remember all the hype about "push" technology back in the mid-nineties? Nobody was interested, but RSS feeds are being used in much the same way now. I'm thinking there are two significant differences: 1) with RSS, the user feels like they're in control of what's going on; with push, users felt like they were at the mercy of whatever money-grabbing corporations wanted to throw at them, and 2) a hell of a lot of people now have an always-on Internet connection with plenty of bandwidth to spare. When you've got a 33.6kbps dialup connection, you use the Internet differently than when you've got DSL or cable.
How much bandwidth does Slashdot's RSS feed use?
It looks like the RSS feed on my home page has a small handful of subscribers. Neat.
$x='S24;r)>63/* h@<5+oZ)32"5cz';$me='phroggy'x$];
$x=~y+ -xz+\0-Tx+;print$_^chop$me for split'',$x;
And institute jackboot banning policies if you access them more than x times per y hours.
One thing that would help immensely is if RSS readers/aggregators would actually cache the RSS feed and not download a new copy if they already have the most current one. I could go through my server logs and point out the most egregious problem aggregators if anyone's interested.
How am I supposed to fit a pithy, relevant quote into 120 characters?
What you're seeing right now are teething troubles. Nothing more, nothing less. The bandwidth and consumption experienced right now will be laughed off a couple of years from now as miniscule.
Take the BBC News website for example. On September 11th 2001 its traffic was way beyond anything it had experienced to that point. Within a year or so, it was comfortably serving more requests and seeing more traffic every day. Proof if it was needed that capacity isn't the issue when it comes to Internet growth, and won't be for the foreseeable future.
RSS is in its infancy. Just because people didn't anticipate it being adopted as fast as it has been that doesn't make it "doomed". By that rationale, the Internet itself, DVDs, digital photography, etc are all "doomed" too.
"Accept that some days you are the pigeon, and some days you are the statue." - David Brent, Wernham Hogg
depends on your perspective. If I imagine myself to be a server, I'm pushing information to a client and pulling information from a client, like the name implies.
you're interpreting it from the client perspective, which is not where the name came from.
[move
And the funny thing here is, if RSS had-- at its conception-- included caching and push-based update notification and all the other smart features that would have prevented this sort of thing from becoming a problem now
What are you talking about? RSS had caching built-in from day one - it uses HTTP as its transport mechanism.
Frankly, I fail to see the point of this article. It already mentions that there are significant ways around the bandwidth problem. Any decent non-polling technique such as pubsub offers a solution, not to mention the fact that virtually everybody complaining about RSS bandwidth use hasn't bothered implementing best practice - things like 304 Not Modifed, Cache-Control headers, etc.
So the executive summary of this story is: some people are complaining about RSS bandwidth. Here are links to solutions. Oh no, sky is falling because I just have to include an ominous yet completely unsubstantiated prediction about next year.
Seems like bittorrent, or a bittorrent-alike protocol would be useful here. Turn the RSSfeed into a tracker/seed and then all it has to keep track of is who has the latest version of the content and it could redirect feeders to each other, always preferring the latest updated version. Eventually, you will have the same scaling problems that bittorrent has (single tracker), but at least you stretch things out a few months or a year until a better solution ocomes around.
I wonder if advertising has anything to do with it - if you go to a news site just to see "what's up", you might get banner ads, google ads, so on and so forth - but RSS just makes a nice neat webpage for you or something similar.
/., Groklaw, and a couple others and it's very cool. Very very cool. I hope the advertising revenue doesn't hurt people or whatever, but it's almost one of those things that would be worth money in how much time and aggravation it saves you having to deal with web designs that aren't as great as they could be.
./ or Groklaw or whatever experience BETTER, not worse.
I have to point out how much I love "Sage", the Mozilla Firefox plugin for RSS - you can even rightclick on that XML thing that tries to tell you to save the page and bookmark it under "Sage Feeds" and then Alt-S and you have your RSS.
I started using Sage for
I've heard a lot about how people complain about Slashdot and the interface and the web design and so on, but Sage cuts down significantly on the time spent here, more or less - or anywhere, for that matter - I think it make the
Only thing I can think of is advertising revenue.
Slashdot user GaryM posted a related question elsewhere about 20 months ago. At that time, in that forum, commenters dismissed his proposed solution, the use of NNTP, on the grounds that NNTP is deficient, but others continue to see NNTP as a possible solution nevertheless.
A lawyer & digital forensics examiner. Also an expert on open source software (OSS).
I'd be interested in seeing how many of these hits are for complete feeds rather than If-Modified-Since the last time it was downloaded. I suspect that if the RSS readers were behaving like nice User-Agents, we wouldn't see such reports.
Perhaps particularly offending User-Agents should be denied access to feeds. If I saw particular User-Agents consistently sending requests without If-Modified-Since, I'd ban them.
My car gets 40 rods to the hogshead, and that's the way I likes it!
RSS feeds are meant as a way to strip all the nonsense from a site and offer easy syndication, right? Basically, present the relevent news from a full-fledged webpage in a smaller file size? If such is the case, this isn't an RSS issue, really.
RSS has different use patterns to normal website visits.
If you visit a website on a daily basis, you might pull down a single hit. You might lose interest after a couple of days. You might not visit at weekends.
But if you subscribe to the feed (RSS, Atom, whatever), chances are, you'll be requesting that feed every hour for as long as you have your mail client/newsreader/web browser open. And even if you lose interest, a lot of people will still remain subscribed and just skim over what doesn't interest them.
Newsfeeds make visitors "sticky". Normally, that's a good thing, but it's actually far better at doing so than need be, "capturing" visitors that really aren't all that interested, and inducing normal visitors to "visit" far more frequently than usual.
That's the inherent difference between serving, say a cut-back XHTML Basic document, and serving an RSS feed that people can subscribe to.
"Slashdot's RSS traffic, like Boing Boing's, is huge, and blocking broken readers has saved us a ton of bandwidth, which of course means money."
So's using correct HTML, and CSS.
I wouldn't doubt that eventually someone will build a RSS caching device & sell it to the corporate market. Given how big a drain as RSS is to the supplier, the corporate market has the money and determination not to permit it to become a problem for them.
Chip H.
Not really. Our cache hit rate would be about zero. We update the homepage about once a minute, and the same goes for any page that any reader would be likely to reload within a reasonable time.
It was meant for syndication. So that one website could gather syndicated news from other sites. It was not meant for individual readers to use it as a news update service. Simply using an appropriate protocol would solve this problem, but do to the blogtard community, this will never happen. And so RSS is doomed to be used stupidly just like it is now.
i'm not sure the overhead of maintaining a torrent would be less than just serving up a single rss feed (or webpage, image, whatever small file). if i'm not mistaken, each client still needs to download the torrent from the main site to determine where it should download the payload from... and if you're going to do that, you might as well just serve up the small file.
also, using a torrent might not work so well for sites like slashdot, which allows users to customize the homepage and/or feeds...
--
I assume the complainers are using it?
51894b boingboing.rss.xml
17842b boingboing.rss.xml.gz
No, I did not read the f***ing article!
Too bad podcasts are totally different from normal RSS feeds, because podcasts are about 100x larger. BitTorrent doesn't work for normal RSS feeds because they are too small and change too often.
I've seen many RSS URLs pull from a site's database to build the XML each time it's hit. This is fixed simply by creating a CRON job that builds the RSS XML on a periodic basis, then serving the resulting file. If you're just throwing a file back, then server bandwidth isn't as much of a problem, especially when you consider that browsers themselves cache files.
Steve Magruder, Metro Foodist
That's XML for ya!
the best way is to optimize your rss feeds to a max of 10 items, and stick to TITLE and LINK fields only.
Tom's hardware had a feed that was over 500kb, and they wonder why they had bandwidth issues.