Slashdot Mirror
Psychologist Beating Math Nerds in Race to Netflix Prize
s1d writes "An almost-anonymous British psychologist named Gavin Potter has suddenly risen to the top of the Netflix prize charts. With his very first attempt, he got a score which took the BellKor team seven months to reach. Currently at a score of 8.07, he has only five teams ahead of him now in the race for the ultimate Netflix algorithm. 'Potter says his anonymity is mostly accidental. He started that way and didn't come out into the open until after Wired found him. "I guess I didn't think it was worth putting up a link until I had got somewhere," he says, adding that he'd been seriously posting under the name of his venture capital and consulting firm, Mathematical Capital, for two months before launching "Just a guy." When he started competing, he posted to his blog: "Decided to take the Netflix Prize seriously. Looks kind of fun. Not sure where I will get to as I am not an academic or a mathematician. However, being an unemployed psychologist I do have a bit of time."'"
And yes, I have karma to burn. Yes I do.
It is a miracle that curiosity survives formal education. - Einstein
The topic is incredibly fascinating. And just a thought, up to 5 centuries ago "scientists" were incredibly versatile people, with mastery over a few fields at a time... A lot of people argue that this was out of necessity, but could the versatility have been important development of multiple renaissances (In Greece, East/West Asia, and Europe)? And could the bottleneck specialization of fields that has occurred in the past three centuries simply be period of transition/stifling new ways of thinking? Could the emergence interdisciplinary experts lead to another 'renaissance' of sorts?
My extensive retail experience says people like to shop by the following methods. Ratings, Genre, Alphabet.
So, if I was to setup a movie viewing for them. I'd setup something along the lines of a Genre, rating(R,PG-13,G), Alphabet.
It's kind of a takeoff on my video game organization method that increases sales of video games by 30 percent. I called it ABSRG short for Alphabetize By Section(4 foot section), Rating(M on top T in the middle and E towards the bottom.), Genre(Sports, driving, shoot em up). Please note, this cannot be patented, I already let it go out for more than a year(Started in 1998)and I have the pictures and time notes to prove it.
I've also found this to be true. Lol, I actually knew people in college that did nothing but program computers in their spare time, and took psych because it was easy, wouldn't distract them and gave them more time to do the programming they wanted. They didn't ever expect to practice psych.
Beware of bugs in the above code; I have only proved it correct, not tried it.
Okay, I'm not trying for this prize, but there's one thing about Netflix "recommendations" that bugs me so I'm throwing out this complete freebie of an idea. If it helps someone get a 0.001% improvement to add this ONE little additional check, great.
I am learning Japanese. I have been watching several hundred Japanese-language movies for the past couple years. I don't watch movies in Greek, Spanish, Turkish, Farsi, Italian, Russian, German, or Hebrew. I did watch Amelie four years ago but that doesn't mean I love French movies. Most of my recommendations are for foreign films, but only a small fraction of those recommendations are for Japanese movies.
Apparently, Netflix doesn't have a column in their database saying WHAT language a movie uses principally, it just has a flag saying it is not English. It's the only explanation I can see for not checking for such a strong correlation. I admit, I might not be sharing the experience of the most common movie-renting drone in the bunch, but I doubt I'm the only person who has such a lopsided taste in movies. If the language (or alternate soundtrack languages) ARE known in the database, please see if the renter has a bias for movies in a particular language.
[
Yes, they did. Every man has a choice between his work and his social life. Many men, great men, choose to dedicate themselves to their work. For the vast majority of these men we call them "geeks". For a smaller number, who end up making the mistake of getting married, we call them bastards. Einstein was one such bastard. He treated his wife and children terribly - and not just because he was basically a German man - but because his work came first.
Besides which, sex as described by the kind of people who think geeks really should be trying harder to get it is basically sport. Geeks have enough common sense to recognize that sport is no fun.
There a joke that I heard (that's actually pretty much true):
A Physicist goes to a Mathematician for advise on solving a Differential Equation. The Physicist explains this and writes the equation on a Blackboard. The Mathematician stares at the equation for more than half an hour. Finally, he says, "Yes, it has a solution."
Basically, the Maths (even applied) are about details and considering them *very* carefully. With this in mind, is it any surprise that they are somewhat "slow"? Especially when they are starting from scratch within the problem domain?
Exactly what I thought, even if he is beating math-nerds, you cannot create a machine-learning algorithm without using fairly sophisticated mathematics. Since psychology is largerly based on statistics, I am sure the guy has a firm grounding in the subject. I am sure he isn't basing his algorithm on Aedipus Complexes (sic) and ink blots.
Aaah, but many psychologists would argue that dedication to work is simply a means to an end. Basically you become important, rise to the top if you will. And then pull a Bill Clinton. The question is not "why would a great man be adulterous and risk losing his position." in the eyes of many behavioral researchers but "Why not? That was the point of gaining the position in the first place." Note: I don't personally ascribe to this viewpoint but it's hard to argue with if you believe that there is no absolute moral code which then implies a creator/supreme engineer of some kind.
Anything is arguable by psychologists. However, if you don't need to believe in an absolute moral code, you also don't need to believe in an archetypical behaviour pattern to which we all adhere (read: behaviour pattern that dictates everything we do is part of some elaborate mating ritual). As such, it may be the case that there are many different reasons why people dedicate themselves to work.
In the case of great scientists, artists, politicians or inventors, it may simply be about pure fascination with their particular interest (eg. Darwin studying and classifying barnacles for 8 years - 1856 to 1864), an altruistic (read instinctive if you will) desire to make living conditions better for their counterparts (eg. Norman Borlaug, 1970 Nobel Peace Prize Laureate, for his research in wheat agriculture leading to the Green Revolution), or some other reason.
In short, perhaps not everything is about f-cking, however f-cking certainly is everything to some.
shaunjohnston.com
Subjects get so complex and so specialized there HAS to be limits to how many giants a human can climb and still have time enough to become a giant themselves. Abstraction helps to greatly extend this range; the people behind abstractions/simplifications may not be considered giants because they do not produce progress themselves but just facilitate others so they can extend their reach into the unknown.
There does not appear to be that many 'giant' scientific figures anymore despite the exponential scientific growth. Maybe it is just an appearance and there are more; but are there more proportionally to the number scientists?
How many big leaps in knowledge have been made in the old fields like physics for example? If the decline does not exist now, won't it exist at some point??
Democracy Now! - uncensored, anti-establishment news
You'd be surprised how many people in psychology are math-phobic and really only know the basic stuff they need to do research (z tests, t tests, ANOVA, regression, power for grant requests, etc) without any deeper understanding of the underlying concepts. If the guy went through a decent quantitative program then he'd have a much better understanding of math since they usually prefer the applicants to have several levels of calculus plus a handful of actual statistics courses (and not stat for psyc), along with the stuff he learned there.
Every post I make begins with the assumption P=~P.