Slashdot Mirror
Interest Still High In the Netflix Algorithm Competition
circletimessquare brings us an update to the status of the million-dollar Netflix competition to develop a better algorithm for movie recommendations. We've discussed aspects of the competition since it started two years ago, but the New York Times has a lengthy overview of where it stands now.
"The Netflix competition is still going strong, with a vibrant, competitive roster of some 30,000 programmers around the globe hard at work trying to win the prize. The Times provides a look at some of the more obsessive searchers, such as Len Bertoni, a semi-retired computer scientist near Pittsburgh who logs 20 hours a week on the problem, oftentimes with the help of his children. There's also Martin Chabbert in Montreal: 'After the kids are asleep and I've packed the lunches for school, I come down at 9 in the evening and work until 11 or 12.' The article gets into the history of the search algorithm Netflix currently uses, and explores the hot commodity called 'singular value decomposition' that serves as the basis for most of the algorithms in competition."
The problem with the Netflix prize, and I myself am working on it :-) is that it is pretty darn near impossible to do better than what they have.
It is based on user ratings and how close you can come to actual user ratings. For instance, their record set has a frozen point in time, you job is to create a system that will accurately predict what another person will rate a movie in the future.
It doesn't take much psychology to understand that these are very subjective values. If you watch a movie on a "good" date, you'll rate it higher than if you watch the same movie with a "bad" date. Then there's the level of drunkenness under which you watch the movie. The day you had at work. How much money you lost in the stock market, etc.
In aggregate, you can come close, but the percentage of variability in the data suggests that Netflix chose their numbers well enough to never have to pay the prize.
Also, the "data" is nothing more than movie titles and obfuscated user ratings. Any sort of contextual or meta data about the movies you have to go find yourself.
It is a fun project on which to work, but I'm dubious of the end prize. I'll keep working on it because its fun, but I have my doubts as to the winability of the contest based on the criteria for success.
Actually Netflix closes nothing off. In fact, in order to receive the prize, the winner must publish their algorithm to the public. The winner could easily open-source the entire thing, or OTOH they're also free to patent it out the wazoo and start pimping it out. The only condition Netflix imposes is that Netflix gets a non-exclusive license to use the algorithm in exchange for the prize money, which is eminently reasonable.
main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}