Slashdot Mirror
Web Geniuses Or Web Dimwits?
ScribeCity writes "The Washington Post has a provocative piece about online experiments at identifying experts. One wonders when someone will come up with a truly effective formula for measuring human intelligence — or take a stab at doing so — that exploits all the stuff people are publishing online." From the article: "This wisdom of the crowd could be outsmarted by what Michael Arrington, editor of the TechCrunch blog, recently dubbed the 'wisdom of the few.' Sites like PicksPal rely on input from the masses chiefly as a venue for auditioning prospective experts, on the theory that these virtuosos could provide even more accurate information and predictions than the crowd. 'If you figure out which ones did the best and get rid of the ones who have no idea, you'd do even better. Distill it down to the people who really know,' Arrington said."
See the book by Philip E. Tetlock called "Expert Political Judgment: How Good Is It? How Can We Know?" by Princeton University Press. He collected the data you are looking for.
Yeah, I sort of thought the whole idea behind 'wisdom of crowds' is the fact that you aggregate enough data to cancel out the individual biases and result in a relatively accurate conclusion. If you pull out all the "experts" who make the correct call in one trial, don't you lose the correcting power of the group? How big and intellectually diverse does a 'crowd' have to be?