Slashdot Mirror
Using AI To Filter RSS Feeds
holden writes "According to a blog post, AideRSS has moved from closed to open beta. I've been using AideRSS over the past few weeks to filter my RSS feeds (including Slashdot and Reddit) and I've been quite impressed. They talk a bit about how the filtering system works, which apparently tracks a mixture of things, from pick-up in other blogs, to some clustering technology."
I'm not sure if it is bad form to comment on your own story, but here goes anyways :). You can take a look at the scored version of the slashdot RSS feed
here, or del.icio.us or my
(holden's) blog. There is also a really cool widget I've put on the side of my blog which lets people subscribe to only posts of a
certain quality (you can look at it here).
It should read more like "AideRSS fianlly released" or "AideRSS goes live."
As for the article, what kind of person or group has too many RSS feeds to look through?
I'm asking because I really have no idea. I have linked the RSS bar in my Gmail to Tomshardware and Slashdot, but that's about all that I need....
www.purevolume.com/martyd
From TFA: ""Some of that data we show on the site itself: Technorati, del.icio.us, etc. Essentially, we're interested in measuring the 'social engagement' of each post. To make this a little less hand-wavy, I think we'll agree that a bookmark is nice but a comment involves more work, a trackback even more so, etc. - hence, engagement). Once we have all this data, we apply our 'secret sauce', which comes in a form of statistical analysis with respect to the author's previous history/posts. PostRank is not a global score, it's with respect to the blogger him/herself.""
;-)
Secret sauce? Why do I prefer open sauce?
One other way to filter RSS is by geographic location through using GeoRSS. However, the source RSS must be offered in GeoRSS for this geolocalization filtering to work... but it's only a matter of time, we'll get there. (hey, even slash has a plugin that works for publishing GeoRSS)
Animoog.org
because the slashdot janitors' javascript programming skills are even worse than their perl programming skills and their editing skills.
If only they could get the AI to do the work I'm missing out when I'm reading RSS feeds.
Anyone know where you can download it?
The secret sauce tastes like teen spirit.
There are some companies out there(i.e. http://www.collectiveintellect.com/) that are using AI to mine RSS feeds and specifically the blogosphere, and selling that data to corporations for various reasons.
Lets say you're a drug company that is releasing a potentially controversial drug. You can mine the data of the blogosphere and issue press releases as a pre-emptive strike to larger media stories. This starts the real beginning of being able to effectively monitor and even potentially control some of the social aspects of the internet. I think it's a great innovation indeed, with potentially scary side-effects.
Personally it is nice to be able to filter through a billion RSS feeds to find information that I'm interested in though.
All things are subject to interpretation, whichever interpretation prevails at a given time is a function of power and n
- The referring website (and what other people from that site liked)
- Your OS/Browser (and what other people with your OS/Browser liked)
- Your geographic location (and what other people close to you liked)
- What you yourself read
It also allows users to edit stories, a mechanism conceptually similar to a wiki, but with an additional voting process to help prevent abuse.Unlike AideRSS, Thoof isn't an RSS aggregator, rather users submit stories, in a manner similar to Slashdot, Digg, and Reddit.
What if the 'other blogs' they 'pick up' on, are in turn using AideRSS to determine what to blog. The whole blogging thing really does seem like one giant feedback loop with only a few people generating actual useful content.
I.O.U One Sig.
I suppose in these modern days when natural resources are being rapidly depleted by overpopulation and overconsumption, there had to come a time when we would start running out of intelligence... of course I wouldn't know because I'm a little short on it myself...
however
It is pleasing to see that scientists around the world have started to produce artificial intelligence to make up for the loss of natural intelligence, but I think that like everything else, perhaps it is also equally important that we conserve and recycle the little natural intelligence we have left and refine our methods to efficiently extract and use that intelligence to, uh, do something or other, but do it efficiently and without any needless waste. Yes, that's my point.
And to that end I see this Artificial Intelligence RSS Feed Filter as a great marvelous invention, because you see, it combines the old and the new, it uses artificial intelligence to extract natural intelligence efficiently and use it for something in a wonderful postmodern fashion. Now, modern invention assists the primitive natural.
Now, all we need is to have a massive SETI like project running this AI RSS Filler Feeder to search for signs of intelligence on slashdot. Oh, oh, cross my fingers, I hope my post makes it pass the filter...
Yes, but can it filter dupes?
I don't know anyone named Al.
Way back in 2003 I wrote some codes to do something similar. I called it Intelli-Aggie and the code is released under GPL. It remains a developmental prototype as I got side-tracked.
IA works, as noted in the readme, by computing a relevance factor, which in turn is based on four other 'relv' - category relevance, feed relevance, keyword relevance and item relevance. I used it as my reader for quiet some time before moving over to 'better' readers.
... "follow me" the wise man said, but he walked behind
I vaguely remember somebody saying the whole point of RSS is that you never get content you don't want because you have to subscribe to it in the first place. What's stopping us from unsubscribing instead of filtering?
The term "Intelli-Aggie" sounds like an oxymoron to me. Hint: Just ask any Longhorn, and he will tell you there's no such thing as an intelligent Aggie.
Why not download this RSS tool instead. It has no T-Shirts but it does filter dupes and put the feeds into a SQL database.
http://www.nullwhore.com/sux0r/index.php?c=/0/logi n/ :-)
http://sourceforge.net/projects/sux0r/
What I find interesting is, it is one of the verrry rare examples of 'internet 2' service that you can own yourself (instead of registering here or there for more ads or worse).
A downside of Sux0r is it seems not having evolved for a couple of years (but still works, possibly that's why
I for one am desperately waiting for a *local* RSS agregator which would allow *me* (and not some site's AI) to Bayes-filter my selected feeds. I'm almost sure this will happenn sooner or later.
Herve S.
Come on. I hear AI I think, you know, self-aware. Couldn't we call it something more appropriate like "smart" or "learning"?
Calling everything and my grandma's toaster AI is getting a little old.
I think there are basically two kinds of RSS Feeds, either they show the latest news (last in first out) or they show an already sorted frontpage (e.g. "crowdsourced" like Digg); both are useful.
Using an AI to resort those feeds is definitely interesting from a coders point of view but trying to give some kind of objective view to a feed is probably not what the average user wants.
Why not do it the other way around and personalize them instead? Maybe it has been done before, but it would be nice if there was a reader to rerank (or even filter out) certain domains, keywords, tags and categories. It could take the given rank as the base score and then resort it according to the user's personal preference, e.g. if someone doesn't like politics he could give the keywords "Bush, Cheney, election, etc." a negative mulitplier and maybe the keyword "funny" gets a positive one. It could even consider the time of the day - politics in the morning and funny pictures during the lunchbreak or something.
Just a qick thought though, someone can perhaps come up with something better. Anyway, I am pretty sure that personalization is the better approach here.
No, then a few people would be generating useful content.
Dare to dream!
B) Eliminate all the stupid users. This is frowned upon by society.
I view with alarm the increasing use of "artificial intelligence" to filter, screen, or otherwise judge human-generated material. In this case it's not enormously important, but it's part of a growing trend.
The issue is lack of responsibility or accountability, because at a certain level of complexity, it is no longer practical to understand or explain the basis of individual decision. The company can just say "the computer did it."
A few years back there was serious consideration being given to using neural nets or something like that to make judgements on loan applications. IIRC the proposed way of handling some sort of legal issues regarding accountability was to add to the system a subsystem that would automatically test the effect of hypothetical changes in the applicant's income. Thus the company could always say "this application was rejected because the applicant's income was too low, and would have been accepted if the applicant had earned X thousand more a year." Raising the question, of course, of whether this was the real reason. Or what it means to talk about "the real reason" in the case of a decision made by a neural net.
In the case of a neural net made of meat, it's possible to cross-examine the net and attempt to find out whether illegal bias played a role in the decision. In the case of an AI neural net, there may be bias built-in... but there's no way to ask the neural net itself about this, and unless the programmers did it deliberately and consciously and left a paper trail, it's pretty hard to find out about it.
"How to Do Nothing," kids activities, back in print!
Finally, I am one of those people who are swamped by news feeds. Some of the feeds I subscribe to are updated very regularly (the news ones) and I don't need to read everything that appears on them others (personal blogs) are infrequently updated and I want to read everything.
Two things I'd like to see:
An offline version; I know it's unlikely to appear (Web 2.0 business model and all that) but I'll never use the online one in the long term.
The ability to upload a bookmarks file filled with rss links. I don't want to have to manually upload all my rss feeds. Also it'd be nice to be able
to change the story levels for all of the feeds from the one page (radio buttons and a table?) rather than having to access each feed before setting the story level.
D.
What's Al Gore got to do with this besides inventing the Internet, and how can I get him to filter my RSS feeds?!
Every time you ________ in Soviet Russia, kitten kills God!
But AideRSS filtered the post out...
So basically the situation would be unchanged.
If I don't put anything here, will anyone recognize me anymore?
I had all my RSS on Google Reader it'd be good to have AI filtering on it too
I am probably an exception to the rule, but I just counted up the different individual news feeds in my NewsReader's (Default-RSS OWL Java OS) start-up OPML file, and there are 1074 unique feeds in it. Granted, there are a few major Mainstream News site feeds that I just recently updated the feed lists from their RSS pages, and haven't yet filtered out the many I consider to be irrelevant from them. Two I recently rescraped but haven't filtered are are McClatchy News and the NYTimes, but even after filtering, the number will still be around 1000.
There are many different reasons for using RSS Feeds. It's more than just keeping up on your favorite web pastime sites. I usually have anywhere from five to ten Google news feeds for specific current event topics I am interested in. For any who dislike the default number of 10 articles that the GoogleNews generated feeds provides as default, add a new parameter argument at the beginning of them (after the ? in the RSS feed URL):
where {varX} is the number of different headlines desired. I haven't checked, but would not be surprised to find an upper limiter on the number, and 100 would be my first guess.
I keep an eye on the feeds provided for open source software packages I use, for notices of upgrades, bugfixes, security issues etc.
I keep an eye on the wonkage produced by many Tanks/Policy orgs via RSS Feeds. I scan tanks/org I generally agree/disagree with, as well as those I consider to be POV neutral.
There are a few very specific feeds to aid in my work and personal free-time collaboration projects.
I desire to acquire as much data input as my mind is capable of assimilating. The issue is avoiding buffer overflows. Any good feed filtering software available that I can use with a high degree of confidence in its results is appreciated, and I am always curious about untried packages.
Rush Limbaugh is a perfect real world example of an oxycontinmoron