Netflix Prize May Have Been Achieved

MadAnalyst writes "The long-running $1,000,000 competition to improve on the Netflix Cinematch recommendation system by 10% (in terms of the RMSE) may have finally been won. Recent results show a 10.05% improvement from the team called BellKor's Pragmatic Chaos, a merger between some of the teams who were getting close to the contest's goal. We've discussed this competition in the past."

Comments

i was joking, however

[circletimessquare]

from the excellent nyt article about the competition in november:

http://science.slashdot.org/article.pl?sid=08/11/22/0526216

it isn't bad movies that are the problem, taste in bad movies can still be uniform

the real problem is extremely controversial movies, most notably Napoleon Dynamite

http://www.imdb.com/title/tt0374900/

not controversial in terms of dealing with abortion or gun control, but controversial in terms of some people really found the movie totally stupid, while some people really found the movie to be really funny

movies like napolean dynamite are genre edge conditions, and people who apparently agree on everything else about movies in general encounter movies like this one and suddenly dramatically differ on their opinion of it, in completely unpredictable ways

Re:i was joking, however

[bill_mcgonigle]

Yeah, all the recommendation systems where I've bought or rented movies, and most of my friends all said I needed to see 'Fight Club', so I did, and ... meh.

Consider this list of movies I've bought/rated highly:

  12 Monkeys
  V for Vendetta
  Lost in Translation
  Donnie Darko
  A Beautiful Mind
  Dogma

I might be grouped with folks who enjoy flicks about identity, man vs. man, those who aren't easily offended, etc. But there doesn't seem to be as clear a way to find a group of people who find aggression offensive, which is basically the driving theme of Fight Club. Perhaps given enough negative ratings it could be possible, but even though I've clicked 'Not Interested' on all the Disney movies, they keep suggesting I want their latest direct-to-DVD crapfest, so I'm left to assume they're rating mostly based on positive ratings.

Re:i was joking, however

[Pollardito]

Perhaps given enough negative ratings it could be possible, but even though I've clicked 'Not Interested' on all the Disney movies, they keep suggesting I want their latest direct-to-DVD crapfest, so I'm left to assume they're rating mostly based on positive ratings.

A co-worker gets almost no recommendations at all from Netflix, and customer service told him that they generate recommendations based on ratings of 4 or 5 (though you'd think that the recommendations that they do generate would have to filter through similar movies that you've rated at 0). He was told to rate the movies that he likes higher in order to fix it, but that's never really accomplished anything as he has several hundred movies in the 4-to-5 range and maybe a dozen recommendations total.

I'm pretty sure that the Disney/children's movie recommendation flood that most everyone seems to be getting is driven by parents who don't actually love those movies, but are rating those movies on behalf of their children. That causes a weird connection to movies that they themselves enjoy, and it makes it seem like the same audience is enjoying both types of movie. They need to have an "I'm a parent" flag somewhere to help them sort that out

Re:Interesting

[Anonymous Coward]

You think using a Generic Algorithm could help?! are you kidding??!! :-)

The search space is far too great and what would you actually be searching for with this technique? (just curious)

I would hope to see techniques evolved from an energy variant of Markov Decision Processes! now that 'would' be a nice direction.

Re:Well done!

[Wildclaw]

Actually, this email has been sent out

"As of the submission by team "BellKor's Pragmatic Chaos" on June 26, 2009 18:42:37 UTC, the Netflix Prize competition entered the "last call" period for the Grand Prize. In accord with the Rules, teams have thirty (30) days, until July 26, 2009 18:42:37 UTC, to make submissions that will be considered for this Prize. Good luck and thank you for participating!"

Re:real world

[Wildclaw]

~0.85 points (on a five-point scale)

Actually the scale is not 0-1-2-3-4 but 0-1-4-9-16 as they use Root-Mean-Square. Just thought it was worth pointing out.

Film recommendations

[michuk]

Does anyone find Netflix recommendations any good anyway? I used http://criticker.com/ for quite a while and was very happy about the recommended stuff. Recently switched to http://filmaster.com/ (which is a free service) and it's equally good, even though both probably use a pretty simple algorithm compared to Nextflix.

Re:Film recommendations

[jfengel]

Reasonably good, actually. I often add 4 star movies to my queue, and rarely regret it.

The problem is the bell curve. There aren't a lot of 5 star movies out there, and I've seen them. There are a lot of 3 star films, but my life is short and I don't want to spend a lot of time on movies I merely "like".

In fact, it's not really a bell curve. I rarely provide 1-star or 2-star ratings simply because it's not at all difficult for me to identify a film I'm going to truly hate. I don't have to waste two hours of my life to find out whether I'd merely dislike the new Transformers movie or whether it will fill my soul with disgust.

The left side of the curve is actually quite fat with movies that simply won't interest me at all. The existing algorithm is actually fairly good at telling me I won't like them. The hard part is picking out the very few movies that ARE worth my time.

They do show both the average and expected rating for each film. What I'd really like to see is a list sorted by the difference: where do I stand out from the crowd? Such movies are likely to have extra appeal.

So the 10% difference isn't completely worthless, but the real problem is that they're pursuing the wrong goal. There's a lot of information they're dropping on the floor.