Slashdot Mirror
Netflix Prize Contest Ends, Down To the Wire
suraj.sun updates us on the Netflix Prize now that the competition has officially closed. We discussed the new leader with one day to go in the contest: The Ensemble, taking the lead from long-time leader BellKor's Pragmatic Chaos, the first contestant to submit an entry that broke the 10% barrier. In the contest's final day, BellKor re-took the lead with 20 minutes to go, then The Ensemble apparently pulled a Michael Phelps with 4 minutes to go, squeaking ahead by 0.01%. At least so the leaderboard claims — but those numbers are posted by the competing teams. The NY Times reports that an official winner will not be named until September — Netflix needs that much time to pore through the complex entries and read the code. Netflix contacted BellKor on Sunday to tell them the team remained in first place; The Ensemble has had no such notification.
Its not about the algorithm for movie sorting. Imagine situations where having a 10% more accurate guess would actually count for something important. Now imagine licensing and patenting that algorithm and building revenue from that.
Support the EFF and Creative Commons. The war is coming, and they're supporting you...
Netflix calculates the score shown on the leaderboard from a set of rating predictions submitted by a team. The team does not, and will not, know the correct answers. For testing their algorithms, the teams use another dataset. The two datasets, part of the package made available to the competitors, are known as "qualifying" and "probe".
The reason BellKor is still first is that the published scores are irrelevant. The scores that matter for the prize are based on an unpublished data set known only to Netflix (to prevent people submitting answers that are optimized for the challenge data and work poorly on everything else). On this secret data set, BellKor's algorithm apparently performs better than The Ensemble's.
main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}
The contest has been going on for several years straight, and /. has had several stories about it. The article takes knowledge of the contest as a given.
See Wikipedia and Netflix's own site for details.
Where have you been?
[2009-07-26]New Leader In Netflix Prize Race With One Day To Go
[2009-07-26]Netflix Prize May Have Been Achieved
[2007-11-27]Anonymity of Netflix Prize Dataset Broken
[2007-11-14]Close but no Cigar for Netflix Recommender System
[2006-10-02]Build a Better Netflix, Win a Million Dollars?
[2008-11-22]Interest Still High In the Netflix Algorithm Competition
[2009-10-09]Netflix Prize Competitor Already Beats Netflix
etc..
No, that link you posted to a web comic we've all seen a hundred times is not "obligatory."
Not exactly. Team A makes algorithm improvement Z Team B hears about Team A's work and creates something SIMILAR. They then fine tunes it to Algorithm Y Team C hears about A & B's work. They try and duplicate it, but can't. Instead they fine they come up with X, something they thought was Y but really wasn't. Team D hears about A,B and C. They find a way to combine Y and X, making W. It is clearly original, but is based on similar ideas for Z/Y and X.
excitingthingstodo.blogspot.com
They used root square mean for the competition.
Basically, the difference between the guess and the real answer for each vote is squared giving a value between 0 and 16 (as the biggest error is 4 when you guess 5 on a vote that is 1 or vice versa). This is summed up for each vote in the test and then divided by the number of votes in the test. Finally you take the root of that.
The winner score in the competition is around 0.855. Which is smaller than 0.9514*0.9 score. Where 0.9514 was the result scored by the netflix algorithm.
I hope that explains everything.
I get this same behavior with every desktop video player I've used. The flaw is with Windows, not Netflix, not Silverlight, not VLC or any other video player.