Netflix Prize Competitor Already Beats Netflix
Baldrson writes "Within the first week of the announcement of The Netflix Prize a team has already beaten Netflix's own movie recommendation algorithm. This is pretty impressive given the previously quoted researcher who said: 'You're competing with 15 years of really smart people banging away at the problem.' The team is WXYZConsulting.com apparently registered by a data mining professor named Yi Zhang. Congratulations are in order for Netflix and Prof. Zhang's team who are demonstrating, yet again, the power of prizes to accelerate progress."
I think this demonstrates how important "many eyeballs" are in problem solving. Intelligent people "who have been attacking the problem for 15 years" can still fail to see an "obvious" solution. I shudder at how many scientific fields probably have obvious solutions that aren't being found because only a small cadre of people have been exposed to the problem. I also shudder at people who artificially set up barriers to understanding their own fields, in order to protect their own egos. The attitude of "journal articles need to be cryptic or they must not be important" needs to go.
Apology to Ubuntu forum.
This reminds me of something mildly entertaining. I used to work at an IT consulting company for small and medium businesses. Each consultant had a short list of "their clients" that only they would take care of. One of my co-workers had a client that was having some problems he couldn't fix. After many other co-workers attempted to fix the issue and failed, our boss offered a $100 bonus to the first person who could fix the problem. Of course, after that incentive, I was motived to look at the problem, and quickly found the it was so stupid that all it would take was a 5 minute fix. I fixed it and collected the bounty. Had he not offered the $100, I probably never would have looked at it. Not a great attitude on my part, but it wasn't my problem and had no incentive to do my co-workers work otherwise.
I think it also demonstrates how the oft-used mantra of "if it needs to be done, it will be done" doesn't always work without some incentive. One of the hurdles of OSS is that the only things that get worked on are the things that people want to work on. The love of developing software can only get you so far (and wow, has it gotten us far). But for some things to advance, it will need financial backing. It's a prickly problem for the OSS community.
My beliefs do not require that you agree with them.
At most companies, the CIO would not let it happen because of the political fall-out that woudl ensue, not because they wouldn't recognize that other people have good ideas as well.
..
The fact that Netflix is allowing customer data out of their control (albiet sanitized data) is a major step that many company's would never take out of reasons not related to the technology at all.
And most CEO's don't challenge those internal assumptions not because of a lack of business sense, but again, because of political savy. The higher you go in a company, the greater the importance of poltiical acumen. It's not even mostly about business. It's mostly about political positioning.
Folks can argue all day long that it SHOULDN'T be about that, but in the real world it is. And frankly, good CEO's and good CIO's need to survive year to year too .