Using AI To Filter RSS Feeds

holden writes "According to a blog post, AideRSS has moved from closed to open beta. I've been using AideRSS over the past few weeks to filter my RSS feeds (including Slashdot and Reddit) and I've been quite impressed. They talk a bit about how the filtering system works, which apparently tracks a mixture of things, from pick-up in other blogs, to some clustering technology."

Comments

Potentially scary side-effects already.

[gethoht]

There are some companies out there(i.e. http://www.collectiveintellect.com/) that are using AI to mine RSS feeds and specifically the blogosphere, and selling that data to corporations for various reasons.

Lets say you're a drug company that is releasing a potentially controversial drug. You can mine the data of the blogosphere and issue press releases as a pre-emptive strike to larger media stories. This starts the real beginning of being able to effectively monitor and even potentially control some of the social aspects of the internet. I think it's a great innovation indeed, with potentially scary side-effects.

Personally it is nice to be able to filter through a billion RSS feeds to find information that I'm interested in though.

Re:Potentially scary side-effects already.

[dada21]

There are some companies out there(i.e. http://www.collectiveintellect.com/) that are using AI to mine RSS feeds and specifically the blogosphere, and selling that data to corporations for various reasons.

Sounds like a fantastic market, actually. I recently picked up a client in the casino management market because I had made some comments on a blog regarding their lack of insight towards proper marketing and keeping a decent percentage of return customers. They actually contacted me, and I've spent a large amount of time parsing through literally hundreds of RSS feeds covering different search terms (the company, the company's competitors, key words of the market, etc, etc). They're always impressed when I hit them up to 50-100 new blogs, news reports, and websites condemning or supporting various new tactics or old dogs. I think that selling companies information that can help them (fix or spin) the problem is a huge market waiting to be tapped properly.

Lets say you're a drug company that is releasing a potentially controversial drug. You can mine the data of the blogosphere and issue press releases as a pre-emptive strike to larger media stories. This starts the real beginning of being able to effectively monitor and even potentially control some of the social aspects of the internet. I think it's a great innovation indeed, with potentially scary side-effects.

I'm not sure I'd call any of the side-effects scary, though. Honestly, I hit Google Blogsearch and Technorati before I hit Google News. What is most important to me, though, is a personal rating system so I can give weight to certain information providers that have given me good information. Lately, I've had better luck from blogs than from the mainstream media. That's a nail in the coffin for the MSM if they don't move with the speed that today's news readers desire.

The benefit of the "free market of information" on the web is that you can now see way more than the black and white sides of things. A drug company may put a spin on bad news, but the issue isn't just "bad news" and "good news." There might be 3 or 4 degrees of separation between the actual negative news and what others may say that could have a positive effect. With the growing (and shrinking?) anonymity of the web, whistle-blowers also have a chance to get the word out -- but again, without a positive history of truth, they may not have a high ranking with me.

Personally it is nice to be able to filter through a billion RSS feeds to find information that I'm interested in though.

I've been sorting through thousands of RSS feeds for a few years now, and have been using some collaborative filtering sites and systems to try to give weight to what I consider the better news sources. Collaborative filtering will be a necessary element to an AI filter because what matters to YOU personally may be missed, but if you use a collaborative ranking, you will also gather information that is important to people who have similar views/needs that you do.

I find a lot more power in the collaborative filtering market than in the AI market. The downside of collaborative filtering is that it can be gamed (see Digg). The upside is that metamoderating of other collaborators can work to fix the gaming individual at a time.

Another site using AI

[Sanity]

Thoof (disclaimer: its my website) uses Bayesian analysis (you could call it AI, so much as anything is AI) to determine what you are interested in reading, based on a variety of factors, including:

• The referring website (and what other people from that site liked)
• Your OS/Browser (and what other people with your OS/Browser liked)
• Your geographic location (and what other people close to you liked)
• What you yourself read

It also allows users to edit stories, a mechanism conceptually similar to a wiki, but with an additional voting process to help prevent abuse.

Unlike AideRSS, Thoof isn't an RSS aggregator, rather users submit stories, in a manner similar to Slashdot, Digg, and Reddit.

Re:Download URL

[forgotten_my_nick]

Someone can correct me if I am wrong but it looks like your supposed to put your feeds into the website then link to the one feed there.

Seeing as half my feeds are internal work related and the fact I don't want someone profiling all feeds I am reading I won't be using the service.