Slashdot Mirror

Validation is overrated. Especially, when it comes to RSS. There's so many competing "compatable" standards, that really aren't. feedparser.org has a great write up about the state of RSS. It's pathetic.

If you're reading a doc, don't bother validating it. You're probably going to have handle "invalid" XML anyway. When you're constructing XML, you should write it according to the DTD, but if you're relying on a remote site, then you're asking for trouble. Just cache the version locally, but seriously, you're tool shouldn't really need it. You're engineers do, but not the tool.

Finally, it's trivial to reconstruct a dtd from sample documents.

MS Patent covers a central system that will be responsible for aggregating various feed formats (RSS, Atom, RDF) and provide a common interface to other programs for using the feed information. Availability of several feed formats and the errors in them makes a nightmare for the applications that need to use feeds. So a common system will be helpful. Several parsers available today to parse RSS and Atom feeds. The most popular one is Universal Feed Parser parses all known web feeds and presents, feed data in usable form that can be used by the application developers. UFP is part of popular open source feed based applications like Planet Feed reader and Democracy Player. So Microsoft's patented process is nothing new. Most of it can be claimed as prior art.

Also Read
Niall Kennedy's wonderful analysis of the MS feed patent.
Blog post by Microsoft Program Manager Lead for RSS Sean Lyndersay defending this patent.

The problem isn't that they didn't validate the user input, so much as that validating user input is really, really hard. RSS aggregators are discovering the problems with validating that HTML is safe. See http://www.feedparser.org/docs/html-sanitization.h tml
The trouble is that an approach like that limits what you can do too much: http://www.symphonious.net/2006/09/10/stripping-st yles-as-part-of-sanitation/
Any site that wants to support formatted comments, like Slashdot, has to deal with this. The plus side when it's just comments is that you can have very draconian limitations to what's accepted without it getting in the way (like Slashdot does).

Except the two nerds mentioned are old-school *nix hands. One of 'em helped invent XML, and plugs Solaris aplenty. The other's written a very good "universal" feed parser that does a pretty good job with all sorts of cruddy feeds, along with a nice free book about Python and another about web accessibility. They don't fit the profile you describe. If you read their reasons, they're dropping OSX because the basic usability gap between it and Ubuntu isn't wide enough (to them) to justify Apple's lack of support for preserving their data.

To be honest, the RSS vs. Atom thing is a lot like DVD+R and DVD-R - at this point they might as well be interchangeable.

Just about every feed parser handles both Atom and RSS feeds. Using a tool like Magpie RSS (PHP) or the Universal Feed Parser (Python) the format of any given feed is entirely transparent to application developers. RSS 1.0? RSS 2.0? Atom 0.3? It all gets processed by the parser in a nearly identical way.

Already tools like Movable Type/Typepad or WordPress generate both RSS and Atom feeds by default. The vast majority of users don't know and don't care which feed format they're reading so long as it works. Both the toolkits and the applications use both formats and there's really little reason why they can't continue to support both.

There doesn't have to be a single "winner" in the syndication feed wars. Atom and RSS can exist together for some time, and arguing that this is a zero-sum game in which one and only one feed format can exist is ridiculous. As long as the difference is transparent to end users, and relatively transparent to developers, neither format will totally conquer the other.

The trouble with RSS (short answer) is that there are at least three different versions of it invented by different people.

It's easy to see why developing a robust feed parser is a real challenge (albeit a necessary one, to hide the current standards chaos from users).

I'm developing an rss aggregator using the Python Universal Feed Parser http://www.feedparser.org/. During my work I've been banned by slashdot as many other developers.

You claim that /. would not benefit from caching because the cache hit ratio would approach zero. However, the modified field in your rss feed, or more correctly the dc:date field, is as far as I have been able to see, NOT updated every minute..

Hence, it appears to me that having your server accept conditional http get should save some bandwith. If nothing else, it may reduce the amount of users you are forced to block.

Am I totally missing something or do I make sense?

Lars Brenna, Norway.