Algorithm Rates Trustworthiness of Wikipedia Pages

paleshadows writes "Researchers at UCSC developed a tool that measures the trustworthiness of each Wikipedia page. Roughly speaking, the algorithm analyzes the entire 7-year user-editing-history and utilizes the longevity of the content to learn which contributors are the most reliable: If your contribution lasts, you gain 'reputation,' whereas if it's edited out, your reputation falls. The trustworthiness of a newly inserted text is a function of the reputation of all its authors, a heuristic that turned out to be successful in identifying poor content. The interested reader can take a look at this demonstration (random page with white/orange background marking trusted/untrusted text, respectively; note "random page" link at the left for more demo pages), this presentation (pdf), and this paper (pdf)."

Comments

7 years???

[Anonymous Coward]

I've been noticing some of the edit histories for articles that are 5 years old on Wikipedia stop well before 5 years ago. Were some of the edit histories been lost or deliberately truncated?

Doesn't take into account common myths

[Cryophallion]

So, if there is a myth that a lot of people believe is true, then it will stay up there as it is not challenged. So, it still gets reputation, and therefore more credibility, making it more likely that the myth will be perpetrated.

Also, if someone hasn't noticed something that is wrong on an esoteric entry, it will also be given credibility, and once again be more likely to be considered to be fact.

While you could add voting to the algorithm to have people vote on whether it is true, that still gets destroyed by someone who just votes because they think it's true, not because they have verified it.

Either way, it potentially gives additional credibility to something that may be very wrong.

Unpopular but neutral points of view?

[Knuckles]

I realize that an encyclopedia by definition will always emphasize the established majority opinion about any given subject. But it seems that this tool might strengthen majority opinions beyond what is reasonable. If you happen to edit an article by adding valid but unpopular dissenting points of view, and the other contributors are sufficiently boneheaded, you lose karma (or whatever the tool calls it) for no good reason. This might then easily develop a life of its own, and you are screwed.

Re:Light Bulb Moment

[marcello_dl]

Sounds crappy. Let's say you expose some important misdeed. You're likely to be edited out by an army of paid staff who keeps an eye on the 'net. (don't tell me I'm paranoid because i saw it happening and read about stuff like that in the news, even slashdot). You are not contributing much else to wikipedia because you simply wanted to expose what's in your knowledge, so you'll end up with a low karma.

Anyway, i guess it'll be another pagerank or slashdot filter affair. People trying to beat it, devs trying to make it better.

The plus is, there is not only wikipedia. You can always search the rest of the web.
The minus is, you search the rest of the web with google which is equivalent if not worse.

We need a good search engine on top of a tor network, and bandwidth to make it run smooth. Not many other way to achieve real net freedom.

It's progress over edit counts

[Animats]

One big problem with Wikipedia has been that editor status, and promotion to "adminship", is based on edit counts, the number of times someone has changed something. The editors with huge edit counts don't generally write much; it takes too long. Slashdot karma is a more useful metric than edit counts, but Wikipedia doesn't have anything like karma.

I'd suggested on Wikipedia that we needed a metric for editors like "amount of new text that lasted at least 90 days without deletion". This UCSC thing is a similar metric.