Netflix Announces Second Data Mining Contest

John Snodgrass writes "Neil Hunt, Chief Product Officer at Netflix, has announced on the Netflix Prize Forums that they are planning to hold a new data mining competition. The second competition will have some twists and is expected to be shorter in duration. It will feature two grand prizes, to be awarded in a 6 and 18 month time frame. A previous competitor still active on the board has already dubbed it: 'The Sparse Matrix: Reordered' and 'The Sparse Matrix: Factorizations.'"

Comments

Contests

[Anonymous Coward]

The people that compete in these contests are weird to me. Although I guess not much weirder than other people that compete in games of chance.

Why do tens or possibly hundreds of thousands of dollars worth of work just for the chance that you might get payed? It seems absurd.

Re:Contests

[Anonymous Coward]

Because it is a lot of fun -- even with no chance of being paid.

Re:Contests

[Smidge204]

> Why do tens or possibly hundreds of thousands of dollars worth of work just for the chance that you might get payed? It seems absurd.

Challenge and notoriety.

For that matter, just about everything you do has a chance of failure, so why do anything?
=Smidge=

Re:Contests

[TheRealMindChild]

Most of the time people want to tinker/play with concepts. They just need some sort of motivation to get them going. The chance that you might get payed is apparently enough for some people.

Re:Contests

[Anonymous Coward]

Then do it on your own or part of an open group. Don't do it for some commercial entity that will screw you and everyone else over to make a buck.

Re:Contests

[Helios1182]

Most of these are research groups that would publish their results and research anyway. This gives them a practical application and a chance for some fame and money -- the research still gets done and published.

Re:Contests

[Anonymous Coward]

Japanese motivation: You might get killed, or at least seriously embarrassed.

Re:Contests

[Helios1182]

I'll add one more thing. Netflix has done the community a favor by providing a large dataset for testing algorithms. Data mining requires data. It requires more than just raw data. It is really difficult to know how well your algorithm works without data that has known answers to compare to. A good test dataset lets you compare your results to other results.

Re:Contests

[Dishevel]

People who take on challenges just because they can? I do believe that they seem "weird" to you. It would seem that way to many lazy people who just want cash for their clunker and to be left aloe to play Halo.

Re:Contests

[timeOday]

For that matter, just about everything you do has a chance of failure, so why do anything?

Obviously the odds do matter. At one end of the extreme is communism, where everybody gets the same regardless of what they accomplish. At the other extreme is "winner takes all," where somebody gets everything, even if they are only 0.01% better than the field. Between those extremes is quite a lot. Which is most motivating? I wouldn't claim to know, and I'm sure it depends on the situation, but it's an awfully important question to society. And I'm pretty sure neither extreme is the best answer (even the Olympics have silver and bronze).

Re:Contests

[Anonymous Coward]

http://wsu.edu/~brians/errors/payed.html

Re:Contests

[Anonymous Coward]

http://wsu.edu/~brians/errors/payed.html

From link:

If you paid attention in school, you know that the past tense of âoepayâ is âoepaidâ except in the special sense that has to do with ropes: âoeHe payed out the line to the smuggler in the rowboat.â

I'm going to need a citation to clarify that this contest actually has to do with ropes.

Re:Contests

[Anonymous Coward]

Read "Longitude" and be grateful you have a watch that works. Your opinion will matter more when it's an informed one.

Re:Contests

[Trepidity]

It allows the researchers to "cheat" a bit too via an argument by authority, which is not always good, but does at least make the researcher's job easier. A big issue in data mining is that it isn't purely a technical field, but one with both conceptual and technical issues. The over-arching goal is something like, "get useful and/or interesting information out of data". But what is "useful", what is "interesting", and how do we measure when we've gotten it or not? Usually you have to defend why your problem is the right one, why your metric is the right way to measure success on it, etc. Working on the Netflix competition lets you sidestep all that, because Netflix has already decreed exactly what the goal is, and what performance metric will be used to judge success at that goal, leaving only the technical problems.

Re:Contests

[Anonymous Coward]

One of many answers is that there are actually a lot of contests, and the concepts and technology used for them overlap. For example, for much lower payouts but higher probability of success, students can enter the UC San Diego data mining competition, among several others.

The concepts and technology needed to do well at these contests also overlap with non-contest applications, like finding employment or consulting work in data mining. Even if you don't win, a respectable result is a useful way of demonstrating that you understand the fundamental issues and how to make progress on them. Many companies aren't looking to hire researchers to push the state of the art in data-mining, so being the winning team (which usually involves some new research) isn't necessarily the only positive outcome; rather, many want to hire practitioners who can apply the current state of the art intelligently, which even a mid-range respectable result is good evidence of.

Re:Contests

[blhack]

Why do tens or possibly hundreds of thousands of dollars worth of work just for the chance that you might get payed? It seems absurd.

Perhaps you have become lost on these internets.

I suggest trying this website.
or perhaps this one.

You will likely find them much more aligned with your interests than slashdot.

Re:Contests

[Helios1182]

I agree, but I still think it is still useful. I see three main requirements for data mining research: data, algorithms, evaluation criteria. (Note: I don't do data mining myself, but know many people in the field and have studied it some).

There are lots of algorithms, but they cannot be evaluated in a vacuum. In fact, the algorithms used tend to be highly customized and tuned for any specific problem. Really, the data, algorithms, and evaluation are a package deal.

Getting all of the necessary components is a lot of work, and I think having the data and evaluation criteria provided by an outside source has some merit. If each researcher defines "good" differently it is impossible to be objective in comparison. Also, how is having Netflix state the goal any worse than a research project leader stating the goal? If I say "predict tomorrow's weather based on the previous 3 days known weather patterns" I have given the project parameters.

Obviously Netflix isn't, and shouldn't be, the only one setting goals for the community; but I think this is win-win for everyone. Netflix gets a new algorithm, customers get better service, researchers get money and acknowledgment. It would be interesting to see other companies or agencies promote such work.

Re:Contests

[Eternauta3k]

lazy people who just want cash for their clunker and to be left aloe to play Halo

Well, to be fair, aloe does help with the chaffing after 48 hours of non-stop Halo.

Re:Contests

[Trepidity]

Yeah, I agree with that. I probably shouldn't have used a pejorative-sounding word like "cheat", even in scare-quotes. I meant just that it lets the researcher get for free some of the things they'd usually have to argue for. From a researcher's perspective, this is a real win: there are many technically solid papers that get rejected from conferences because the reviewers thought the problem wasn't interesting enough ("maybe I believe you solved this, but why?"), or the metric wasn't the right one. Nobody's going to reject your Netflix-prize-related paper for those reasons.

It does even provide a good jumping-off point for questioning those assumptions, so I agree it's not a bad thing in any way for Netflix to be proposing goals like this. There have been papers, for example, that accept the basic ratings-prediction goal of the competition, but argue that the specific performance metric used doesn't capture the high-level goal that well.

Re:Contests

[Anonymous Coward]

You should spend more time crafting really, really good trolls, and then searching for relevant posts / stories to use them on... because your trolling is kinda boring right now. I guess, in time, you will get better.

As the one who made up the names

[Anonymous Coward]

I have to say that until September, I will wake every morning with the same question: "What is (or will be) the Matrix?" LOL

Don't take me seriously.

CS1

Re:As the one who made up the names

[Anonymous Coward]

It didn't take you long to crow about this on Netflix's Prize Forum.

Usefullness?

[deserted]

It's a neat contest, but I'm really not sure how it helps their customers at this point. We're talking about the decision-making process human beings go through to decide how they wish to be entertained. I favor some movies just because I enjoyed them as a whole, and their algorithm will not be able to be granular enough to figure out exactly why I like something. On the other hand, having a suggestion system is very helpful. I'd would be pleased to be reminded of a summer comedy that I perhaps forgot about. However, there just aren't enough movies out there that I would skip over one had it not been for their algorithm.

Re:Usefullness?

[Anonymous Coward]

As someone who used to watch 3 movies a day for about 3 years straight, I still found the system to be useful.

I thought I'd seen everything that was worth watching but if you're really dedicated to finding more quality films then any help is good help, and this is one of the better systems for finding new films (more accurate than trawling imdb but maybe not quite as fun)

Re:Usefullness?

[sarlos]

Another factor affecting an algorithmic approach is the mood of the viewer. While this isn't as important when it comes to the mailed DVDs, it plays a huge role when considering their streaming content. I may pick Movie X to watch because the wife and I each had a hard week, but Movie X may be something that we'd never view under any other circumstance. A discrete system has a very hard time categorizing something as fluid as mood and could easily be led to make very inaccurate recommendations on the whole.

Re:Usefullness?

[KCWaldo]

The ratings apply to more than just movies. They can be used for anything can be rated. Music, toys anything.

Re:Usefullness?

[mixmatch]

Well, I'm certain they could easily incorporate a 'What kind of movie are you in the mood for today?' question and a dropdown for the different options....

Re:Usefullness?

[sottitron]

I used to think I was unique in what rare movies and music I liked until I met someone who had almost an identical collection to me. On top of that, we both had some of the same clothes. The reason netflix researches these data mining techniques is because our tastes really do cluster into groups. For some it might be because they like DeNiro films and Spaghetti Westerns. For others it might be that they like two screen writers - though they never know it. The payoff for getting this right if you are Netflix is that if a customer gets recommendations and he/she use those to fill your queue, then they are much less likely to cancel netflix anytime soon.

Re:Usefullness?

[Helios1182]

Apparently recommendations are important, otherwise they wouldn't put that much money towards it. There are tens of thousands of movies you have never heard of, but chances are you might like some of them.

Re:Usefullness?

[Bakkster]

I may pick Movie X to watch because the wife and I each had a hard week, but Movie X may be something that we'd never view under any other circumstance. A discrete system has a very hard time categorizing something as fluid as mood and could easily be led to make very inaccurate recommendations on the whole.

If it has a hard time categorizing it, it's because you gave it bad data with your ratings. If the movie's a one-off thing, either don't rate it, or rate it down.

That said, a sophisticated rating system should be able to recognize multi-modal distributions. I like some dumb comedies, some cerebral Science Fiction, and some action thrillers. A good system should pick out my trends amongst each of these to make suggestions within each genre, some crossovers, and really wouldn't be affected by the one oddball movie I'll never watch again.

It doesn't seem too out of line to assume that if you watch enough "mood movies", a good system will make several suggestions for the next time your wife is in a mood for a similar movie, as well as suggestions for your movie watching other times. They aren't mutually exclusive. It's just up to you to look at the appropriate "Because you liked Romantic Comedies/Foreign Films/Summer Blockbusters/Erotic Thrillers" suggestion box for the mood you're in. The system isn't just telling you "Watch this one movie now, I know more than you", but it can give you a much refined set of choices.

Re:Usefullness?

[Ceseuron]

I'm not sure what the purpose of these data mining contests are. However, as a member who prefers instant streaming over my XBox 360 over waiting for the mailman to drop off a DVD, I hope the contest yields a better selection of instant playback material. Instant playback on Netflix currently suffers from a mediocre selection of obsolete, boring, B-grade movies. One can only watch Dolph Lundgren's "Retrograde" so many times before questioning whether or not the Netflix membership is even worth it.

Re:Usefullness?

[spongman]

the streaming selection on netflix is limited because netflix has to ink an often complex 'boradcast rights' deal with the studio for each movie in order to steam it. netflix has to compete with other broadcast companies (mostly the tv networks, but also companies like hulu, iTunes), and in many cases the deals are for exclusive rights over a given time period. none of this applies to shipping DVDs.

I believe that adding movies to your 'watch instantly' queue allows netflix to prioritize which movies it should license for streaming.

Re:Usefullness?

[Bakkster]

True, but it's possible for a sufficiently advanced algorithm to guess which movies might have shared tags (without naming them), between yourself and others, and make them recommendations. It could even be better than letting humans fill out the tags (which results in tag-bombing, such as on Amazon), assuming a sufficiently large data set.

Think like this, you like Cerebral Lovestories such as ESOTSM. ESOTM is rated highly by yourself and 49 other people and low or not at all by everyone else. If 25 of those people like another, similar movie (let's call it My Gray Matter) that is not liked by the general population, the algorithm can assume that they are somehow related. Say you've all also liked 5 other Cerebral Lovestories; there is now a significant link. There is a large correlation between people who like Cerebral Lovestories, and a correlation to liking My Gray Matter where none exists elsewhere. Therefor, there's a high probability that My Gray Matter is a Cerebral Lovestory, without the need for tagging and much more protected from abuse.

Re:Usefullness?

[SatanicPuppy]

Everybody thinks they're unique. Really though, the range of human behaviour isn't all that wide. You can think of groups of people like circles in a venn diagram...Even very different people can have a great deal of overlap.

Re:Usefullness?

[im_thatoneguy]

Then again watching a diverse selection of films isn't terribly advantageous to them. While keeping you happy is concern #1, offering a diverse and perhaps unusual selection isn't any better than keeping you happy with a string of blockbusters.

Re:Usefullness?

[Neoncow]

According to these guys, movie data's usefulness recedes as more sophisticated data mining algorithms are implemented.

Since they are part of the winning team, there's a good chance they're right. (They could also be lying about it to throw off the competition, but I believe they are required to publish their method so we will find out.)

Yes, this is an appeal to authority, but I only did it because the authority in question claims to have access to strong evidence.

Human reaction machines. . .

[Fantastic Lad]

There's nothing at all wrong with studying how the human automatic processes work, but "Psychology for Prizes" does have a very Neil Stephenson feel to it.

The public eagerly jumping for the chance to teach corporate bodies how to better advertise to them seems a little preposterous. In a world where everybody's objective is openness and self-study for the betterment of humankind, this sort of thing would be laudable, but here it's a bald-faced attempt to fine-tune manipulation techniques.

What would be cool would be if Netflix, upon offering you a suggestion, would also explain what reasoning they used to offer that suggestion to you. Open-source advertising. If every billboard had an explanation of the psychology behind it, we could learn much more about ourselves. The amount of free will that we use every day versus automatic behavior can only increase when the illusion of free will is broken down and examined.

-FL

Re:Human reaction machines. . .

[MozeeToby]

The only reasoning that is used is "You liked [movie group A], other people who liked [movie group A] also like [movie B] so maybe you will too". There may be something in there to make the groupings by genre but I doubt it, when the first contest started Netflix reluctantly made genre information available after a couple teams asked for it.

Re:Human reaction machines. . .

[UCSCTek]

There's a difference between getting people to make impulsive, thus irrational, decisions and providing targeted advertising that might actually be something the viewer wants and was not aware of. The latter arguably benefits the consumer, making people happier. I don't really see how Netflix is doing anything but this. I do agree on opening up ad psychology. Only the manipulative advertising stands to lose.

Re:Human reaction machines. . .

[Fantastic Lad]

There's always going to be an argument which makes a manipulative and self-serving action sound benign and cheerful. I remember watching a news piece about one of the top McDonald's CEO types heading over to Russia to try to establish the golden arches there. In a candid shot, he described McDonald's as a sort of angelic entity whose mission was to bring hungry children meat, bread and milk. I wondered if he had really convinced himself of that horseshit or if it was just a face he put on for others.

It's all about spin. The problem is that when profit is the primary motivator, then you cannot ever trust a seemingly friendly face put forth by a company. They don't want to be your friend. They want you to think that they are your friend in order that you might feel comfortable in giving them your trust, money, time and energy.

Now, if the Netflix guys are actually motivated not by profit, but by an over-riding love of film and the desire to share film with the world, then it's a whole different story. You do see this sometimes. I've known several owners of private shops who really love what they sell, but when you scale things up past a certain number of employees, even a founding love takes a back seat to the corporate need to grow profit share and absorb wealth. It's almost like a company only has a single soul which is shared by every participant in the company and thus gets stretched thin.

-FL

Re:Human reaction machines. . .

[blhack]

I know that I personally *HATE* it when a company can offer me a service that I really want.

Something like this really pisses me off, though, because netflix is coming into my house and forcing me to watch their "ads"...err..I mean "suggestions".
I mean, it isn't like I signed up for the service and pay them a monthly fee exactly because they have a huge library of movies to me! The worst part is that they charge me every time I rent another movie! A move like this is just an attempt to get me to rent more!

Wait...wut?

Oh, yeah, this sort of "the world is ENDING" "corporations are t3h EVILZ!" fearmongering is some of the most refined bullshiat available for consumption on the internets.
How could you even possibly look at this as something bad? They're offering a whole ton of money to some geeks to do what the geeks love doing: tinkering with computer machines.

Re:Human reaction machines. . .

[Knowbuddy]

The public eagerly jumping for the chance to teach corporate bodies how to better advertise to them seems a little preposterous.

Really? Makes perfect sense to me.

You might carry around your Minority Report-inspired retinal-scanning tinfoil hat, worried about the evils that faceless corporations can inflict upon us if they know our buying habits and personal preferences. I'm a bit more pragmatic: they're going to try to make money, and selling me things I want is a pretty good way to do that.

Here's the thing: advertising isn't going away. Yeah, I'd love for the local politicians to get a wild hair and suddenly decide to tear down all the billboards along the highways. But do I think that'll ever happen? MaHellNo.

So instead, why not cooperate with the corporations to at least move advertising to a state where it doesn't make you want to claw your eyes out?

You may not, but I personally love the "You Might Also Like"-type features when done well. I'm a consumer. I buy things. In a world with millions of products, yeah, I could use a bit of help separating the things I might be interested in from the stuff that I won't. Show me something that says "10 of your friends rated this item with 4 stars or better" and I'm going to pay attention.

Movies/films are a great place for this type of work. We've got a history of over 100 years of cinema -- enough that no one has seen everything. Want to help me separate the wheat (Run Lola Run) from the chaff (The International)? Yes, please.

Re:Human reaction machines. . .

[Fantastic Lad]

I'm a consumer.

Yeah? I'm a person.

-FL

Re:Human reaction machines. . .

[treeves]

They're basically the same thing. A Venn diagram would show the set of consumers almost entirely within the set of persons and a small subset of the person set would be outside the consumer set - or they'd be equivalent sets - depending on your definition of "Consumer".

The Need for This Contest

[SEWilco]

The purpose of this contest is to figure out who won the previous contest.

What better way

[RomulusNR]

to take advantage of hordes of unemployed technologists than to get them to provide months of free work?

Re:What better way

[Anonymous Coward]

The unemployed ones are by and large too retarded to be of any use to this contest.

Comment removed

[account_deleted]

Comment removed based on user account deletion

Comment removed

[account_deleted]

Comment removed based on user account deletion

A Portia D/Olivia W luv scene w/2 hr tension bldup

[Impy the Impiuos Imp]

> Netflix Announces Second Data Mining Contest

Oh thank god I've got another chance!

I was gonna solve the previous one challenge, but never quite got around to it.

Advertising value

[cbraescu1]

Seems like the only reason they keep coming up with such contests is their advertising value. Just my 2 cents.

Consumers and People

[Fantastic Lad]

Definitions again. Allow me to clarify. . .

I tend to think that to call oneself a "consumer" is the result of a stupendous and multi-generational maneuver of marketing which reduces the human to the status of a mindless eating machine with no other virtues or qualities of significant value. Sadly, for the most part, this is an accurate state of affairs, but I choose to deviate from that model. I refuse to see my purpose in the world as being simply to desire and work relentlessly towards the acquisition of the colorful array of material goods presented to me by Walmart.

Life, I have discovered, is much, much bigger and far more fulfilling than that. Consequently, I have found endless ways to exist both comfortably and happily without the need to make or spend very much money at all. In this regard, I am a very bad consumer, however I strive to excel in the art of being a person. I have many more hours in the day than most to do with as I please, allowing me to explore the world to my satisfaction in countless fascinating ways. Further, I put my energies to good use; many people consider me a significant asset in their lives.

As I see it, those who accept the mantle of 'consumer' also accept by definition, a very narrow form of existence, --one which serves the world of corporations and governments and not of people and communities.

-FL

Re:Consumers and People

[treeves]

So the connotations of "consumer" lead you to shun its denotation. I can understand that. Alas, as with so many words, once it develops distasteful connotations, resistance is futile.