Slashdot Mirror
Best Practice For Retiring RSS Feeds?
GBJ writes "I work for an organisation that runs seasonal online competition events. Each event has its own news feed which becomes obsolete shortly after the event finishes. We're still getting RSS requests for some events as far back as 2004. I'd like to close a few thousand old feeds and remove the resource hit they cause, but I'm not sure what is the best approach. Currently I'm considering just returning a 404, but I have no idea if there is a better way to handle this. Uncle Google hasn't turned anything up yet, but sometimes it's hard to find something when you don't know what it's called ..."
Retiring feeds
A phase-out needs.
As facial bristles,
Or torn skin bleeds.
Burma Shave
Get thee glass eyes, and, like a scurvy politician, seem to see things thou dost not.--King Lear
You should be able to do a 301 redirect to maybe some generic feed that just has one entry that says "This feed is out of date, please use try these feeds instead." Or whatever you want to let them know.
Reviewing just the first hour of video games.
The most appreciative thing you could do for the preservation of history is to place static simple RSS files at those addresses that include a link to your archives for the event.
IMarv
Trusting software vendors is no smarter than trus
RFC 2616 is the one to read. It specifies a "410 Gone" for resources that are gone for good.
If so, you can either archive the feeds on a web page (of only web page versions) and charge for "back issues" to cover the costs.
Still getting requests? Is it folks who are emailing yo explicitly for the feed or you're seeing traffic on those feeds in you logs?
If it's just traffic, maybe it's just some default setting on someone's phone program or something? In other words, those requests aren't "real".
Either way, I say archive them and if they really want to see them, let them bring up a browser. You're still offering service but you're not, I don't think, taking up much resources.
Gone
Indicates that the resource requested is no longer available and will not be available again. This should be used when a resource has been intentionally removed; however, it is not necessary to return this code and a 404 Not Found can be issued instead. Upon receiving a 410 status code, the client should not request the resource again in the future. Clients such as search engines should remove the resource from their indexes.
use a 410 Gone more informative than 404
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.11
ERR 411[Max number of witty sigs reached]
The resource isn't "Not Found", it's "Gone". HTTP 404 is inappropriate in this instance.
Likely the best solution will be to ensure that people are notified first. If you're receiving a large number of hits, replace the content with a single RSS item that has a guaranteed-unique guid for every single request (say, based on the request time). This way, with each request, people will receive a "new" item that will display as unread, reminding them to unsubscribe from that particular feed.
After some amount of time, start returning HTTP 410 for all requests.
HTTP 410 is better than HTTP 404 in this case; Uncle Google and the like have a better understanding of what it means.
If you have a version of the feed that covers the whole event from start to finish, you might also want to offer a static version of that for download as an archive (but if you do this, put it on a different URL from where the feed used to be). This isn't strictly necessary, but I can see scenarios where people might appreciate being able to get at the feed's contents again.
Are there more than one contest going at once, that there's a need for multiple feeds? Or, more appropriately, can some of this content not be removed completely? Keeping a full feed archive seems a bit of overkill, especially for closed events from five years ago. Why not PDF the event archive for downloading and keep a single feed for active items? Overpreparation is a growing problem I'm seeing on the web. Far too few people/events/businesses are prepared to minimize anything for the sake of optimization.
Those things you're doing with that stuff you just bought? That's not what it's for! -
Just return an unending stream of crap from /dev/random on your server until it crashes the RSS reader at the other end. After a few days of this I'm sure they'll sort it out from their end :-)
Code, Hardware, stuff like that.
He mentioned a couple of thousand old feeds. Either he's been serving feeds since the days when RSS was chiseled by hand into stone tablets, or he has distinct feeds which run concurrently - in which case reusing isn't going to work.
My company has had the same issue. We just wrote out a single item feed that explained that the feed had been discontinued and provided a link to the homepage. We wrote this feed over all the feeds to be taken down.
Interestingly, in the weeks following this action, our homepage views spiked followed shortly after by increased hits on more active feeds.
No way man. You can't just dump all those people like that!
Either redirect them to a feed that says "here are some other events you might be interested in subscribing to" or create a last post on the feed that says the same thing. Dumping them with a 410 is a great way to loose your traffic.
It sounds like in this case, a redirect is the proper thing. Just be forewarned that some readers (cough Google Reader) will redirect, but ignore the "permanent" in "permanent redirect" and will continue to hit the old feed.
Newsreaders are amazingly stupid. Google Reader, for example, will ignore both your "301 Permanent Redirect" and ignore your Atom link="self" as well. Something tells me it will ignore your "410 gone" too. You should test what major newsreaders (MyYahoo, Bloglines, Google Reader) do when you deep-six a feed with a 410.
Even still, 410 is stupid. A 301 redirect to something generic would be better, even if the readers are too brain dead to get the message. 410="lost traffic".
This seems like more of an opportunity than a problem. People haven't deleted the feed, so they obviously still want to hear from you. Redirect requests to the 'dead' feeds to a general feed that announces new events.
You've got one feed for updates in each sport or event.
Imagine horse racing. You are making updates to racing scores throughout the day. Gamblers are monitoring each race. How would you structure this king of thing? One feed per day? One per season?
What about comments? You've got a feed per thread/story. When the story is closed for comments, what do you do with the feed?
What about auctions? You've got an auction and every bid triggers a new entry on your RSS feed. When the auction is over, what do you do with the feed?
Yes. Because if the newsreader flakes out, the *person behind the computer* might blame you and never subscribe to your feed again. We are dealing with people here, not machines. Were are not here to seek spite on stupid newsreaders or "lusers who dont unsubscribe". This is business. Petty bullshit only wastes time and loses money.
Even if the newsreader was well behaved and did the right thing, you've just stupidly removed your brand from being thrown in their face every time they opened their reader. If you've got like 10,000 subscribers sitting on a bunch of year-old feeds and you cruelly dump them with a 410/404, that is lost mindshare and lost traffic.
The best solutions are the ones that encourage feed users to keep using your services by nudging them to your new stuff.
Bottom line is, it is easier to keep existing customers than it is to get new ones. Think about it.
Your best bet for solving the problem you currently have is to have done something different in the past?
That's probably the least helpful advice ever :)
Are RSS feeds really causing an insane amount of traffic? Does that traffic cost more than it does to acquire new subscribers? I doubt it.
Let me qualify that though. Before I did anything drastic like 410'in their ass, I'd come up with the total number of subscribers to the entire pool of feeds. Most of the big-boy newsreaders will usually include a subscriber count in their User-Agent. Tally up the numbers and make sure you aren't dumping 100,000 subscribers in an attempt to save a piddly 1,000 feed-hits a day.
Sounds like you would benefit from a site level "current events" feed, that always has the most recent events. Then a per-event feed that, after the event, changes to "thank you for coming, see more at "current events". And that feed expires 3 months after the event.
If people are collecting your old feeds, but not getting value from them, is that actually valuable traffic? Are you interested in getting the word out, or counting fictional traffic?
You can also rename the feed to archive old information. Renaming the feed breaks the links to lazy readers. Anyone who really wants to research your old event data will search for it starting at your home page.
This is Slashdot, and I guess some here view the world in some kind of mechanical way free from the constraints of the real world. So yeah, if he wants to tell them to fuck off and feed their newsreader streaming crap from /dev/urandom, by all means do so. But that is a child's thing to do. Adults view the world different.
A mature adult would see that the cost of bandwidth is minimal and the untapped potential in all those people sitting on ancient feeds outweigh the potential costs. A mature adult who strives to have a successful website would find a way to tap that potential, possibly by redirecting the feed to something that nudges them to the good stuff.
But no, go ahead and stream your fucking mp3s to their newsreader. That will teach them to ever cross path your mad skillz. It will teach them so good they'll never visit your site again, never click your ads and never buy your services. Who needs their money, right? After all, rent is cheap living in the basement!
(ps: booya)