Slashdot Mirror
Recommendation Algorithm Wants To Show You Something New
Several sources are reporting on a new metric that computer scientists are going after with respect to recommender systems — recommendation diversity. "In a paper that will be released by PNAS, a group of scientists are pushing the limits of recommendation systems, creating new algorithms that will make more tangential recommendations to users, which can help expand their interests, which will increase the longevity and utility of the recommendation system itself. Accuracy has long been the most prized measurement in recommending content, like movies, links, or music. However, computer scientists note that this type of system can narrow the field of interest for each user the more it is used. Improved accuracy can result in a strong filtering based on a user's interests, until the system can only recommend a small subset of all the content it has to offer."
creating new algorithms that will make more tangential recommendations to users, which can help expand their interests,
The advertising industry already has that technology. Their idea of "expand interests" usually involves shopping, of course.
How long before it has enough data to recommend we should be destroyed and acts on it?
So, first we start out with one TV in the house and mass appeal programs. Then, as we get more and more channels, each user watched a specific channel targeted to their demographics. Then we got more specific programs from podcasts, and recommendation systems told us what we'd like before we knew it existed. The problem was, then the content makers didn't know how to sell such small audiences, so we're going to have to muck up the recommendations systems to suit them... sure, good luck with that.
Even small percentage increases in per-order purchases can result in huge gains across the board. Netflix, with a comparatively paltry prize amount, has bought themselves an incredibly efficient revenue generating piece of software.
I'm surprised to see that it still relies on popularity ranking as a cornerstone of the algorithm, but the other areas, especially heat diffusion and random walk are very cool and I'd love to read more about it.
Books. I am an avid reader of sci fi and fantasy, and man, most recommendations out there just BLOW.
Sent from your iPad.
We've been both promised personal recommendations and been threathened with personalized advertisement, yet I hardly ever see any of it.
Take Youtube I thought there was something fancy behind it but now that it displays _why_ it's recommending a clip, you can tell that it's extremely simple. Being a practictioner in machine learning and AI myself, I must confess that most industry implementations in our field is 10% very simple stuff, with 90% boring database and infrastructure code around it.
No news websites allow personalization, Google has (supposedly) only minor tweaks for the individual user. There's the recommendation system at Amazon, true, but it stands out only because it's the only one worth mentioning. (May I have overlooked some music streaming sites here?)
Compared to what we can do with search engines, the state of the art and the implementations are dismal. Is it a Really Hard Problem (TM) ? Consider the Netflix competition. Several groups worked feverishly to improve on the inhouse Netflix recommendation system and did so, by only 10%. Can we really hope for a breakthrough?
Every recommendation algorithm I've seen does one or both of two things. The first being staying extremely close to things I have already expressed an interest in - never broadening my horizons.
That, or it suggests really popular things, for example with music always getting a string of well known, popular bands and artists like Radiohead or Pink Floyd suggested as bands I might like - because many people who like similar sorts of music to me like Radiohead, the algorithm thinks I would like Radiohead too - they can't seem to figure that I would already know if I liked Radiohead or not at this point. I've never found a way to tell a recommendation algorithm that Pink Floyd is OK but I want something less popular...
they are both equally bad... it's just that the people with money chasing the solution are no longer sold on the standard approach... it's quantum now... if we all go for the blonde no one gets the blonde, so recommend hair dye.
This isn't really that new at all. I've seen several other groups doing something with diversity vs similarity in recommender systems.
"However, computer scientists note that this type of system can narrow the field of interest for each user the more it is used. Improved accuracy can result in a strong filtering based on a user's interests, until the system can only recommend a small subset of all the content it has to offer.""
Slashdot: "I see you've subscribed to certain opinions. Here are some more recommendations."
Shai Schticks:"You don't make peace with friends, you make peace with enemies"
Have you ever been in a Turkish prison?
I understand the problem; the direct connection criteria between two different things might be completely indecipherable or insurmountably complex and subtle (let alone indirect relationships i.e. six degrees of Kevin Bacon). That means whatever you build has to account for trends to narrow the band of complexity which leads to the same old problem of only suggesting status quo.
A tool that can only suggest "obvious" or "random" things leads to undesirable results and at best can only fractionally provide you with "success".
I still think it's a cool project with a lot of opportunity for discovery but I just can't get past the idea that you either tell people what they want (advertising) or let people discover things on their own (interconnection).
crazy dynamite monkey
I recommend getting out into the Big Blue Room and doing something real, tangible, and unique!
...he always wants to show you something new, and it's always in the back of his van along with the puppies and candy.
They should use IBM's new algorithm, it's faster than the old one.
One reason this kind of problem occurs is that many collaborative filtering algorithms are measured based on "root mean squared error", basically the square root of the mean of the differences between what was predicted and what the user actually did.
The problem with this metric? It doesn't account for a variety of important things, one of which is that most users value diversity. Another is that in most recommendation systems, what is important is the relative relevance of recommendations to each-other, whereas RMSE is an absolute measure of effectiveness. And a really tricky one is that the recommendation algorithm itself can impact user behavior. For example, the user may raise their standards if the algorithm does a better job.
The unfortunate answer is that the only rock-solid way to measure the effectiveness of recommendation algorithms is to test them with real users, perhaps splitting the user population between different algoritms, and seeing which does best.
I'm pretty familiar with this issue as my day job is building a behavioral ad targeting engine. We learned a long time ago that while RMSE has its uses, there is often limited correlation between an algorithm's ability to predict user behavior retrospectively (which ads they will click on and what products they will buy), and how much additional revenue the algorithm will generate in practice.
Our solution is to use RMSE as a first-blush indication of how good an algorithm is. Secondly, we take the top, say, 10% of ads with the best predictions, and see what the actual click or conversion rate is within this 10%. This requires a higher volume of data, but yields results that are closer to what we find in reality. Lastly, the algorithm then has to prove itself in the wild on a small subset of traffic. Only then can we really know if any algorithm is an improvement on any other.
A system like this can work in two ways. Either the similarity measure computes a "distance" between the music by analyzing the sound and metadata, or it maps a band to a group of people that like it and then maps from this group back to other bands the group likes.
If the system is employing the latter, we have a problem. If we select only popular bands or pick bands randomly, there's no hidden wisdom of the crowd for the algorithm to extract. We can't blame the software, only ourselves
The first example is good, but if you are searching for "The Police", it's unlikely any algorithm, or human observer would think you were searching for the band. If you searched for "The Police Band" (without quotes obviously) then I would say fair enough.
Otherwise it would be like searching for "big black cock" and being surprised that the results and ads were not about poultry.
...some other research groups with names that make me giggle like an idiot.
CAn'T CompreHend SARcaSm?
Recommendation engines, such as the ones used by Netflix and Amazon.com, already recommend really random choices. Except in the simplest cases (you bought a nail gun, you might want some nails), current recommendation engines stink at figuring out what I want. Trouble is, the recommendations are interesting so few times that I don't even look at them any more. I'll bet these researchers actually used to work for Amazon.com creating their recommendation engine, but got fired. Not knowing what else to do, they wrote a paper to describe what they did!
As someone who neither follows, nor particularly cares about this topic, did anyone have a lul or two when they noticed the acronym used in the first line of the quote? "PNAS"
Trolling is a art and for that i give me 3,00 internets.