Slashdot Mirror
Interest Still High In the Netflix Algorithm Competition
circletimessquare brings us an update to the status of the million-dollar Netflix competition to develop a better algorithm for movie recommendations. We've discussed aspects of the competition since it started two years ago, but the New York Times has a lengthy overview of where it stands now.
"The Netflix competition is still going strong, with a vibrant, competitive roster of some 30,000 programmers around the globe hard at work trying to win the prize. The Times provides a look at some of the more obsessive searchers, such as Len Bertoni, a semi-retired computer scientist near Pittsburgh who logs 20 hours a week on the problem, oftentimes with the help of his children. There's also Martin Chabbert in Montreal: 'After the kids are asleep and I've packed the lunches for school, I come down at 9 in the evening and work until 11 or 12.' The article gets into the history of the search algorithm Netflix currently uses, and explores the hot commodity called 'singular value decomposition' that serves as the basis for most of the algorithms in competition."
It's actually not that hard to build an algorithm which works well. Following a demonstration at TechEd I built my own implementation using Python in about 2 hours (using a vector space algorithm) or so with reasonable results. The problem is that it is very difficult to win the prize.
The best thing about it is that you get a lot of data to play with. If you are interested in parallel algorithms and large data sets give it a go. Its surprisingly interesting and sucks you in. In fact I might go play with it now.
Each new algorithm takes on average three or four hours to churn through the data on the family's "quad core" Gateway computer.
Anyone who puts "quad core" in quotes like that is either clueless, or---when talking about Gateways---astoundingly ironic. It's kudos either way!
When Bertoni runs his algorithms on regular hits like Lethal Weapon or Miss Congeniality and tries to predict how any given Netflix user will rate them, he's usually within eight-tenths of a star
Makes me wonder how accurate my own ratings would be. The difference between clicking 3 or 4 stars is often very minor and arbitrary. At the end of a movie I might rate it something totally different than 20min later. Sounds like they're doing pretty good so far.
There's a sort of unsettling, alien quality to their computers' results ... But many categorizations are now so obscure that they cannot see the reasoning behind them. Possibly the algorithms are finding connections so deep and subconscious that customers themselves wouldn't even recognize them.
Realizing the program you wrote out-performs you and you can't explain why is a rather odd feeling.
got from this, even when it has to pay out the prize it will be very cheap against any going rate.
Why is it considered selling yourself short if you do work for free for a commercial entity... but not when you contribute to, say, FireFox, ThunderBird, Apache, the Linux kernel, and so forth and so on?
In both cases you are typically doing work for absolutely zilch as far as cash or prizes go. You may get a fuzzy warm feeling on the inside, you may simply enjoy doing the work (similar to the fuzzy warm feeling), maybe you enjoy the popularity it gives you. On rare occasions, maybe the work you do there lands you a job further down the line but that's not really something you can bank on. In both cases, you are also doing work somebody else -could- have been doing, for actual pay. I won't get into an argument of whether doing work for free means you're 'stealing jobs' - fact simply is that Netflix -is- getting a lot of work done practically for free that they would otherwise have had to hire somebody for; you would have to agree as otherwise "selling themselves short" would not apply.
So yes, you're doing work that should be landing you some cold hard cash when you...
- devise a matching algorithm for Netflix
- create a video for Radiohead
- submit photos to a Canon photo competition that they are then free to use in any and all marketing material aka ads regardless of whether your photo actually won that competition.
But isn't that pretty much the status quo that many here -want- to go to? Those making their money with proprietary programming, creating arts, etc. are dinosaurs in dying business models, no?
(only semi-flamebait)
It's not selling yourself short to work on FOSS for a very simple reason. Work on FF, or Thunderbird, or open-sourcing a script that I wrote to convert music is free at the point of delivery. That is, anyone can use it without paying. Freely given, and freely distributed.
However, in this case the user of the algorithm is paying Netflix. Netflix takes the work that I have done, and closes it off from other people. My work goes not to benefit the community, but merely to benefit one company - a company that has paid me (cheaply) for my work. Since companies by definition only care about the bottom line, their intent is not to benefit the community, but to benefit themselves. You are effectively working for them for cheap, selling yourself short.
If netflix were to give away the algo for use by anyone else too, then it would be very generous and then you may be able to make a comparison with FOSS. I( have no idea if they will do that or not. However, if I were a shareholder, I would not want them to give away a potentially killer feature for which they paid $1m.
Saying that, if you enjoy playing with this, go ahead! Just be honest with yourself about. If you still want to do it, wallow in it. But it's an extremely pernicious thing to do to link this with working on something that is done to benefit everyone. It simply is not the same thing.
The problem with the Netflix prize, and I myself am working on it :-) is that it is pretty darn near impossible to do better than what they have.
It is based on user ratings and how close you can come to actual user ratings. For instance, their record set has a frozen point in time, you job is to create a system that will accurately predict what another person will rate a movie in the future.
It doesn't take much psychology to understand that these are very subjective values. If you watch a movie on a "good" date, you'll rate it higher than if you watch the same movie with a "bad" date. Then there's the level of drunkenness under which you watch the movie. The day you had at work. How much money you lost in the stock market, etc.
In aggregate, you can come close, but the percentage of variability in the data suggests that Netflix chose their numbers well enough to never have to pay the prize.
Also, the "data" is nothing more than movie titles and obfuscated user ratings. Any sort of contextual or meta data about the movies you have to go find yourself.
It is a fun project on which to work, but I'm dubious of the end prize. I'll keep working on it because its fun, but I have my doubts as to the winability of the contest based on the criteria for success.
Actually Netflix closes nothing off. In fact, in order to receive the prize, the winner must publish their algorithm to the public. The winner could easily open-source the entire thing, or OTOH they're also free to patent it out the wazoo and start pimping it out. The only condition Netflix imposes is that Netflix gets a non-exclusive license to use the algorithm in exchange for the prize money, which is eminently reasonable.
main(c,r){for(r=32;r;) printf(++c>31?c=!r--,"\n":c<r?" ":~c&r?" `":" #");}
I've used netflix on video over the internet for a year or two now. The way to solve the problem is to break the star ratings up into a few different categories. You can always leave an "overall" rating for the lazy people, but if someone really wants netflix to "get to know them" they need to be more specific about what they like in the movie.
.... OMG! The special FX were done with PBRUSH, and they used the microphone that was built into the directors handy cam the whole time. Yes, it was that crappy, I actually had to show this movie to other people so they would believe me. I'm not a producer or anything, but I could shit on a paper plate and kick it against a clean white wall, and that would make a better movie. Merc force.... I will never forget you.
Right now neflix tries to infer what it was in the movie you liked by looking at other movies. Why not just ask what they liked about the movie.
For instance, I'm very concerned about the production quality in a movie. The movie may have the best plot ever and great actors but it was shot on a home VHS camera. I would give the movie a 1 star because the production quality was so bad, on the other hand someone who likes plots may have rated it a 5 star. Now netflix will never know if I rated it 1 star because I don't like the genre or don't like the acting or the cinematography. It just sees I rated the whole movie as a 1 and any movies that have similar elements then lose their importance on my personal ratings. If I could tell netflix: don't show me movies shot on a VHS camera (e.g.: production 1 star) then I could tell netflix I love the genre, love the plot hate the production.
A good example is Blood Ryane - this movie absolutely sucks (insert government sponsored movies jab here), but I like the genre - now if I give this one star, as it deserves, netflix will think I really don't like the... whatever, it's most likely going to be wrong about it because it's pure conjecture.
I'm not a big movie nerd so I wouldn't be the best person to come up with the rating categories, but I'll give it a shot since this will never occur:
1. Production Quality
2. Plot
3. Directing
4. Acting
5. Genre
Of course this will never happen because netflix will not change their system to conform to my random idea on slashdot. And by this sentence I've just about exhausted all my interest in the subject.
One last comment: Why are all the online netflix movies so craptastic? Really, if it wasn't made 15 years ago, and it's in the "watch instantly" section, then it must really suck. They had a movie on there called "merc force"
In the same vein, a 55 year old man is going to like a teenaged girl much better than NAPOLEON DYNAMITE.
exploitation is exploitation, self imposed or not; as well, contests are, by their very nature, one-sided, there must be a loser, if there is to be a winner, therefore, competition is only valuable within the framework of co-operation, otherwise, yes, it is exploitative
see? oh well probably not....
I think you don't understand the concept of "fun." Read the article and the comments and tell me that the people "working" at this competition aren't getting paid handsomely. If money is the only compensation that means anything to you, you must be an economist. Congrats, you're doing your part to keep up economics' reputation as the "dismal science."
If it's exploitative for a company to provide enjoyment and intellectual stimulation to a lot of people and benefit financially as a result, then I guess publishing companies don't deserve my support, either. And the movie studios... theater companies... restaurants... and of course any bands that get paid for gigs are just a bunch of ruthless exploiters.
I guess the only commercial entertainment that's okay is what I can enjoy completely passively, without any mental effort at all. That way I'm not being exploited, right? Because work is an awful, awful thing ;-)
Wake up; it's not the nineteenth century or even the twentieth century. Everyone has a natural appetite for work, and unlike our unfortunate ancestors, ours is not overwhelmed and sickened by the work required for mere survival. You're addressing a relatively privileged group of people; we earn enough to support ourselves on less work than we have an appetite for. As a result, we don't have to regard all work as a curse imposed on us by necessity. Work freely done and enjoyed is a blessing.
If you insist that our entire appetite for work be channeled through grim-faced contract negotiations, then that blessing is ruined. What's the point of ruining our fun? So you can save us from the misery suffered by our great-grandparents?
Obviously none of what I said applies to call center employees, game company employees, and technical support employees. They should pay careful attention to what you say ;-)