Algorithm Rates Trustworthiness of Wikipedia Pages

paleshadows writes "Researchers at UCSC developed a tool that measures the trustworthiness of each Wikipedia page. Roughly speaking, the algorithm analyzes the entire 7-year user-editing-history and utilizes the longevity of the content to learn which contributors are the most reliable: If your contribution lasts, you gain 'reputation,' whereas if it's edited out, your reputation falls. The trustworthiness of a newly inserted text is a function of the reputation of all its authors, a heuristic that turned out to be successful in identifying poor content. The interested reader can take a look at this demonstration (random page with white/orange background marking trusted/untrusted text, respectively; note "random page" link at the left for more demo pages), this presentation (pdf), and this paper (pdf)."

Comments

Re:7 years???

[Anonymous Coward]

Some edit histories are also completely messed up in random order. Look at the weird edit history for Wikipedia's article on Pi "Revision as of 21:54, 8 September 2002" precedes older revision "Revision as of 06:17, 5 December 2001"
  How can we trust the Wikimedia software if it corrupts the edit database?

Re:7 years???

[Stooshie]

RTFA!

• The demo is based on the Wikipedia dump dated February 6, 2007. The demo contains pages that are contiguous in the dump; pages were not selected manually or individually. The demo contains the last 50 revisions of each page (or fewer, for pages with fewer revisions).
• Occasonally, the coloring breaks the Wikimedia interpretation of the markup language. We are trying to resolve all such issues by locating the coloring information appropriately.
• The algorithms are still very preliminary.
• No, you cannot edit the pages. :-)

Re:Seems a bit dangerous

[ajs]

Editor wars are an old thing. The real concern I'd have would be how you deal with old editors who don't contribute anymore (but were "trustworthy" when they did) vs. new editors. Overall, I think it's a good idea, and I would go so far as to say that MediaWiki should offer a feature that performs this analysis for you.

-~~~~

Re:Seems a bit dangerous

[ajs]

You're not actually reading the text that they linked to, are you?

We're not talking about Wikipedia's concept of authorship, here, but the tool's. The tool tracks who first wrote something and doesn't re-assign authorship because it was removed (e.g. by a vandal) and then restored.

You would have to remove what they wrote and then restore it in your own words in such a way that your edit was good enough to be retained by the community. In which case, the system worked.

Overall, I think it would be an excellent thing.