Slashdot Mirror
Could a Reputation System Improve Wikipedia?
Acidus writes, "There is an excellent article in this month's First Monday about using reputation systems to limit the effects of vandalism on public wikis like Wikipedia. It discusses the benefits and weaknesses of various algorithms to judge how 'reliable' a given piece of text or an edit is. From the article: 'I propose that it would be better to provide Wikipedia users with a visual cue that enables them to see what assertions in an article have, in fact, survived the scrutiny of a large number of people, and what assertions are relatively fresh, and may not be as reliable. This would enable Wikipedia users to take more advantage of the power of the collaborative editing process taking place without forcing that process to change.'"
The site is /.-ed, but this got me thinking: what about having an additional page view that uses color to highlight text age? Oldest text would be black, newest would be something else (red? blue?), intermediate 'ages' in intermediate shades. This would make it quite obvious which parts of the article haven't been modified in a long time.
-- the cake is a lie
That way if end users want the latest breaking news on an article they could see the latest rev at the risk that someone vandalized it -- or if they wanted a less up-to-date but true version the could look for the last "accurate" tag.
These accurate tags could play much the same role that Debian Stable does compared to Unstable.
There are successful reputation systems out there. For instance, Credence can avoid Gnutella spam. They have a cool algo for detecting when a group of fake users all rate the same bogus files up. Wish this sort of thing was more widely deployed. Bitzi is the dumb version of the same idea, but never worked for me.
Yeah but some of the most brilliant things on the internet might have a low "page rank" on Google. Still, Google's reputation system (which is exactly what it is), does a pretty good job, even with the fact that it must infer ratings by links. Obviously, they have to work pretty hard to make it hard to game.
Of course it was. It was perfectly formed for the best karma payoff with the least amount of effort. That's what posting to Slashdot is all about, right?
The role of the writer is not to say what we can all say, but what we are unable to say. -Anais Nin
A smaller but much less gameable step would just be annotation of recent edits. Yes, you can look them up in the history if you want to now. No, this won't help with people who post believable BS. But it'd be nice if when skimming an article I could see the information that's been there forever and is likely not in dispute vs the information that got posted 3 hours ago by some anonymous jackass.
Of course, while you couldn't game this to make information look more factual (a recent edit would always show up as such) you could discredit information if you wanted by editing it slightly. Also it might lead to people not wanting to correct style/grammar/etc errors for fear of ruining the article's reputation.
About 10 years ago, when I learned of PGP's "Web of Trust" system (where I could choose to trust everybody that you trust), I turned to a colleague and said "What we need next is a 'Web of Credibility'..." where individuals in the community can bestow credibility points to others... and the points I can bestow to you would depend upon how credible the community thinks *I* am, and so on. In other words, if Noam Chomsky or Lester Thurow vouch that you're highly-credible, then that would boost your credibility more than a few glowing scores from your cable guy and the kid at the local sip-n-go. Ultimately, it would be a measure of how likely (or unlikely) you were to spout off on something that you had no clue about.
Having not yet RTFA, I'd just like to say that I agree, wholeheartedly, with the general notion... and I look forward to the day when our credibilities are incoporated into our digital signatures (that I hope we're also all using someday). - Joe
Sometimes it's obvious when a factual error has been made. Say, for example, somebody changes the article on Monarch butterflies to claim that they feed on the blood of human babies. Anybody with an ounce of reason can see that it's a fallacious claim. But what if it's a minor factual error that can easily slip past your notice? Or worse... an outright lie that seems more reasonable than the truth? Say, for example, somebody claims that the volume of a mol of helium is 23.6L. Outside of people with an actual background and education in chemistry, nobody's going to notice that error. Critical thought or no, that's something you just have to know in order to see as wrong. Those with a background in Chemistry know that it's actually 22.4L. Those without have no clue.
And the problem arises when people then use that number as fact, without bothering to do research outside of Wikipedia. A *lot* of research papers, particularly in High School and Undergraduate studies, skimp on research. I know people who have never been to a library to research, and do all of it online. These are the people getting burned. Arguments over the morality or wisdom in doing all of your research in that manner aside, Wiki is fast becoming the prime source of research data. And that's why the people at Wiki are trying to find a way to improve their credibility.
If you believe everything you read, you'd better not read. - Japanese proverb
Because the harder you try, the closer you come to success.
In contrast to most articles on the Israeli-Palestinian situation where the writer is trying to advance a particular viewpoint, I've actually found the Wikipedia articles to be quite good. You actually have a situation where people on both sides of the issue are coming together to try to figure out what the facts are. In my experience, it's the extremists committed to propagating factual inaccuracies that eventually get driven away from wikipedia because, unlike when they preach to their choirs, the audience at wikipedia actually cares whether the articles are factually accurate.
On the other hand, I have found some of the high traffic articles to be overly neutralized. For example, I was interested in the crimminal justice system in Singapore but, when I went to the main article on Singapore, all it contained was bland touristy propaganda. One of the main things Singapore is known for in the world is harsh punishment of minor crimes but there was only one sentence about it buried in an unrelated paragraph way down the page.