Slashdot Mirror
Recommendation Algorithm Wants To Show You Something New
Several sources are reporting on a new metric that computer scientists are going after with respect to recommender systems — recommendation diversity. "In a paper that will be released by PNAS, a group of scientists are pushing the limits of recommendation systems, creating new algorithms that will make more tangential recommendations to users, which can help expand their interests, which will increase the longevity and utility of the recommendation system itself. Accuracy has long been the most prized measurement in recommending content, like movies, links, or music. However, computer scientists note that this type of system can narrow the field of interest for each user the more it is used. Improved accuracy can result in a strong filtering based on a user's interests, until the system can only recommend a small subset of all the content it has to offer."
creating new algorithms that will make more tangential recommendations to users, which can help expand their interests,
The advertising industry already has that technology. Their idea of "expand interests" usually involves shopping, of course.
How long before it has enough data to recommend we should be destroyed and acts on it?
So, first we start out with one TV in the house and mass appeal programs. Then, as we get more and more channels, each user watched a specific channel targeted to their demographics. Then we got more specific programs from podcasts, and recommendation systems told us what we'd like before we knew it existed. The problem was, then the content makers didn't know how to sell such small audiences, so we're going to have to muck up the recommendations systems to suit them... sure, good luck with that.
Even small percentage increases in per-order purchases can result in huge gains across the board. Netflix, with a comparatively paltry prize amount, has bought themselves an incredibly efficient revenue generating piece of software.
I'm surprised to see that it still relies on popularity ranking as a cornerstone of the algorithm, but the other areas, especially heat diffusion and random walk are very cool and I'd love to read more about it.
Books. I am an avid reader of sci fi and fantasy, and man, most recommendations out there just BLOW.
Sent from your iPad.
Most of the time when a site suggests something for me I have absolutely no interest in what they suggest. I don't see how a tangential approach could be any less effective.
We've been both promised personal recommendations and been threathened with personalized advertisement, yet I hardly ever see any of it.
Take Youtube I thought there was something fancy behind it but now that it displays _why_ it's recommending a clip, you can tell that it's extremely simple. Being a practictioner in machine learning and AI myself, I must confess that most industry implementations in our field is 10% very simple stuff, with 90% boring database and infrastructure code around it.
No news websites allow personalization, Google has (supposedly) only minor tweaks for the individual user. There's the recommendation system at Amazon, true, but it stands out only because it's the only one worth mentioning. (May I have overlooked some music streaming sites here?)
Compared to what we can do with search engines, the state of the art and the implementations are dismal. Is it a Really Hard Problem (TM) ? Consider the Netflix competition. Several groups worked feverishly to improve on the inhouse Netflix recommendation system and did so, by only 10%. Can we really hope for a breakthrough?
My PNAS wants to show you something too. Careful or it'll squirt you in the eye.
Every recommendation algorithm I've seen does one or both of two things. The first being staying extremely close to things I have already expressed an interest in - never broadening my horizons.
That, or it suggests really popular things, for example with music always getting a string of well known, popular bands and artists like Radiohead or Pink Floyd suggested as bands I might like - because many people who like similar sorts of music to me like Radiohead, the algorithm thinks I would like Radiohead too - they can't seem to figure that I would already know if I liked Radiohead or not at this point. I've never found a way to tell a recommendation algorithm that Pink Floyd is OK but I want something less popular...
I'm sure I posted this very idea to /. years ago, probably more than once. I'm also sure I'm not the only one who recognizes the utility of joggling the output of a recommendation system.
they are both equally bad... it's just that the people with money chasing the solution are no longer sold on the standard approach... it's quantum now... if we all go for the blonde no one gets the blonde, so recommend hair dye.
This isn't really that new at all. I've seen several other groups doing something with diversity vs similarity in recommender systems.
I happen to like fine wine, vintage automobiles, afternoon tea and Goatse. I am looking forward to meeting others with similar interests.
If I'm searching for digital issues of Kieth Giffen's Trencher comic, I am not interested in targeted ads for earth digging equipment. If I am searching for The Police, the band, I am not interested in law-enforcement training diplomas. These are good test cases to see if targeted advertising is any good or not - all of them I've ever seen fail these cases.
"However, computer scientists note that this type of system can narrow the field of interest for each user the more it is used. Improved accuracy can result in a strong filtering based on a user's interests, until the system can only recommend a small subset of all the content it has to offer.""
Slashdot: "I see you've subscribed to certain opinions. Here are some more recommendations."
Shai Schticks:"You don't make peace with friends, you make peace with enemies"
take
heroin.
Losers.
Yours In Vilnius,
Kilgore T.
I understand the problem; the direct connection criteria between two different things might be completely indecipherable or insurmountably complex and subtle (let alone indirect relationships i.e. six degrees of Kevin Bacon). That means whatever you build has to account for trends to narrow the band of complexity which leads to the same old problem of only suggesting status quo.
A tool that can only suggest "obvious" or "random" things leads to undesirable results and at best can only fractionally provide you with "success".
I still think it's a cool project with a lot of opportunity for discovery but I just can't get past the idea that you either tell people what they want (advertising) or let people discover things on their own (interconnection).
crazy dynamite monkey
I recommend getting out into the Big Blue Room and doing something real, tangible, and unique!
...he always wants to show you something new, and it's always in the back of his van along with the puppies and candy.
They should use IBM's new algorithm, it's faster than the old one.
One reason this kind of problem occurs is that many collaborative filtering algorithms are measured based on "root mean squared error", basically the square root of the mean of the differences between what was predicted and what the user actually did.
The problem with this metric? It doesn't account for a variety of important things, one of which is that most users value diversity. Another is that in most recommendation systems, what is important is the relative relevance of recommendations to each-other, whereas RMSE is an absolute measure of effectiveness. And a really tricky one is that the recommendation algorithm itself can impact user behavior. For example, the user may raise their standards if the algorithm does a better job.
The unfortunate answer is that the only rock-solid way to measure the effectiveness of recommendation algorithms is to test them with real users, perhaps splitting the user population between different algoritms, and seeing which does best.
I'm pretty familiar with this issue as my day job is building a behavioral ad targeting engine. We learned a long time ago that while RMSE has its uses, there is often limited correlation between an algorithm's ability to predict user behavior retrospectively (which ads they will click on and what products they will buy), and how much additional revenue the algorithm will generate in practice.
Our solution is to use RMSE as a first-blush indication of how good an algorithm is. Secondly, we take the top, say, 10% of ads with the best predictions, and see what the actual click or conversion rate is within this 10%. This requires a higher volume of data, but yields results that are closer to what we find in reality. Lastly, the algorithm then has to prove itself in the wild on a small subset of traffic. Only then can we really know if any algorithm is an improvement on any other.
A system like this can work in two ways. Either the similarity measure computes a "distance" between the music by analyzing the sound and metadata, or it maps a band to a group of people that like it and then maps from this group back to other bands the group likes.
If the system is employing the latter, we have a problem. If we select only popular bands or pick bands randomly, there's no hidden wisdom of the crowd for the algorithm to extract. We can't blame the software, only ourselves
...some other research groups with names that make me giggle like an idiot.
CAn'T CompreHend SARcaSm?
We working on quite the same topic approaching from a different point of view and trying to exploit the concept of serendipity to try to obtain similar resuts.
Our first publication on the topic:
http://www.computer.org/portal/web/csdl/doi/10.1109/HIS.2008.25
Piero Molino
http://www.di.uniba.it/~swap/index.php?n=Membri.Molino
Recommendation engines, such as the ones used by Netflix and Amazon.com, already recommend really random choices. Except in the simplest cases (you bought a nail gun, you might want some nails), current recommendation engines stink at figuring out what I want. Trouble is, the recommendations are interesting so few times that I don't even look at them any more. I'll bet these researchers actually used to work for Amazon.com creating their recommendation engine, but got fired. Not knowing what else to do, they wrote a paper to describe what they did!
Screw recommendations, what I want is a better search and organizations. When you look for example at Amazaon they still are to stupid to put all the books in a comic book series together, you have to manually search for each of them or hope that the recommendations picked one of them up. Why don't they just show you a list of everything related to a series? How about a search for all comics in which Spider-Man appeared or movies where two actors played in together? Movies where an specific actor had more then 10min screen time and all that kind of detailed stuff. Why am I limited to a stupid text search over an items description, why can't I search the actual content? On top of that Amazon doesn't even seem to have their advertisment linked with their sales, they constantly recommend stuff hat I already bought from them.
Looking through torrents, release lists, Wikipedia and other third party information is much more informative then what the commercial companies manage to provide and without that extra help it would often be near impossible to find some items and actually buy them.
As someone who neither follows, nor particularly cares about this topic, did anyone have a lul or two when they noticed the acronym used in the first line of the quote? "PNAS"
Trolling is a art and for that i give me 3,00 internets.