Slashdot Mirror
Psychologist Beating Math Nerds in Race to Netflix Prize
s1d writes "An almost-anonymous British psychologist named Gavin Potter has suddenly risen to the top of the Netflix prize charts. With his very first attempt, he got a score which took the BellKor team seven months to reach. Currently at a score of 8.07, he has only five teams ahead of him now in the race for the ultimate Netflix algorithm. 'Potter says his anonymity is mostly accidental. He started that way and didn't come out into the open until after Wired found him. "I guess I didn't think it was worth putting up a link until I had got somewhere," he says, adding that he'd been seriously posting under the name of his venture capital and consulting firm, Mathematical Capital, for two months before launching "Just a guy." When he started competing, he posted to his blog: "Decided to take the Netflix Prize seriously. Looks kind of fun. Not sure where I will get to as I am not an academic or a mathematician. However, being an unemployed psychologist I do have a bit of time."'"
5th place is not winning. Are you guys Ron Paul supporters too?
It's called domain knowledge people. It helps being a psychologist when you're write a program reacting to people's behavior. If programmers knew how to do that, they would get laid more.
And yes, I have karma to burn. Yes I do.
It is a miracle that curiosity survives formal education. - Einstein
that had an undergraduate degree in psychology, a masters degree in operations research that after being well employed for a number of years -- "In 2006, he left his job at IBM to explore the idea of starting a PhD in machine learning, a field in which he has no formal training. When he read about the Netflix Prize, he decided to give it a shot -- what better way to find out just how serious about the topic he really was?"
The summary makes 2 references to Gavin Potter being a psychologist, but it ignores the part of the article that notes he has a master's degree in operations research. This is very much an OR problem. Still, it is impressive that he has been able to do as well as he has considering his competition. Good luck to him!
The topic is incredibly fascinating. And just a thought, up to 5 centuries ago "scientists" were incredibly versatile people, with mastery over a few fields at a time... A lot of people argue that this was out of necessity, but could the versatility have been important development of multiple renaissances (In Greece, East/West Asia, and Europe)? And could the bottleneck specialization of fields that has occurred in the past three centuries simply be period of transition/stifling new ways of thinking? Could the emergence interdisciplinary experts lead to another 'renaissance' of sorts?
He might be a psychologist, but his venture firm is named Mathematical Captital, after all. His partners appear to have advanced degrees involving mathematics.
512 MB RAM, 20 GB disk, 200 GB transfer, five datacenters. $19.95/month.
I don't understand why Wired insists on playing along with Potter's pretense of being an "unemployed psychologist". He's a PhD candidate in machine learning, has a masters in operations research, is ex-IBM and Pricewaterhouse, runs a VC firm -- he has plenty of quantitative and computational training and experience, probably more than most of the contestants.
What I'm listening to now on Pandora...
My extensive retail experience says people like to shop by the following methods. Ratings, Genre, Alphabet.
So, if I was to setup a movie viewing for them. I'd setup something along the lines of a Genre, rating(R,PG-13,G), Alphabet.
It's kind of a takeoff on my video game organization method that increases sales of video games by 30 percent. I called it ABSRG short for Alphabetize By Section(4 foot section), Rating(M on top T in the middle and E towards the bottom.), Genre(Sports, driving, shoot em up). Please note, this cannot be patented, I already let it go out for more than a year(Started in 1998)and I have the pictures and time notes to prove it.
I've also found this to be true. Lol, I actually knew people in college that did nothing but program computers in their spare time, and took psych because it was easy, wouldn't distract them and gave them more time to do the programming they wanted. They didn't ever expect to practice psych.
Beware of bugs in the above code; I have only proved it correct, not tried it.
Okay, I'm not trying for this prize, but there's one thing about Netflix "recommendations" that bugs me so I'm throwing out this complete freebie of an idea. If it helps someone get a 0.001% improvement to add this ONE little additional check, great.
I am learning Japanese. I have been watching several hundred Japanese-language movies for the past couple years. I don't watch movies in Greek, Spanish, Turkish, Farsi, Italian, Russian, German, or Hebrew. I did watch Amelie four years ago but that doesn't mean I love French movies. Most of my recommendations are for foreign films, but only a small fraction of those recommendations are for Japanese movies.
Apparently, Netflix doesn't have a column in their database saying WHAT language a movie uses principally, it just has a flag saying it is not English. It's the only explanation I can see for not checking for such a strong correlation. I admit, I might not be sharing the experience of the most common movie-renting drone in the bunch, but I doubt I'm the only person who has such a lopsided taste in movies. If the language (or alternate soundtrack languages) ARE known in the database, please see if the renter has a bias for movies in a particular language.
[
The dataset is about 660MB to download. It unpacks to 2GB of about 18,000 text files.
What a bunch of drivel. Just because their level of knowledge isn't what we have today, doesn't make it any "easier." Do you have any idea at all, or can you even comprehend, the kind of mathematics that were employed back in the day to solve anything? Take a look at the Principia for example. The geometry is insane. I'm a graduate student in Physics and I can't really follow his proofs.
Furthermore, because early scientists did not have as much to build on, that makes it all the more difficult. Where was Faraday to get his inspiration on lines of force? What lead Maxwell in the right direction to unifying light with electromagnetism?
It's great that 3rd graders know about electric circuits. That's the point of scientific progress. That doesn't make the original task trivial in any sense.
In other words, I hate you.
There are 480,189 customers that rated 17,770 movies. The total number of ratings that you're given is 100,480,507. Each user/movie/rating is accompanied by the date of the rating, as well. You then have to submit predictions for the ratings of 2,817,131 additional user/movie combos (they tell you the user, the movie, and the date, and you need to predict the rating). You submit these predictions to Netflix, and they tell you the root-mean-square error between your predictions and the actual ratings that those users gave those movies.
OK, how do you solve a cubic? Or grind the lenses for a telescope? Or build a water pump? How do you dress a head wound? Or isolate pure gases? Or calibrate a thermometer?
If you know how to do all those things, good for you. My point is that a secondary school education in "science" does not by itself provide one with the same understanding as the great scientists who first among all others figured out how to do each of these things.
Shop as usual. And avoid panic buying.
Could the emergence interdisciplinary experts lead to another 'renaissance' of sorts?
Oh, I don't know. If only we had a art history & statistics dual major to figure it out...
HA! I just wasted some of your bandwidth with a frivolous sig!
Am I the only one that finds the phrase 'An almost-anonymous British psychologist named Gavin Potter' faintly ridiculous?