Could a Reputation System Improve Wikipedia?

Acidus writes, "There is an excellent article in this month's First Monday about using reputation systems to limit the effects of vandalism on public wikis like Wikipedia. It discusses the benefits and weaknesses of various algorithms to judge how 'reliable' a given piece of text or an edit is. From the article: 'I propose that it would be better to provide Wikipedia users with a visual cue that enables them to see what assertions in an article have, in fact, survived the scrutiny of a large number of people, and what assertions are relatively fresh, and may not be as reliable. This would enable Wikipedia users to take more advantage of the power of the collaborative editing process taking place without forcing that process to change.'"

Comments

Easier said than done?

[gasmonso]

I agree that they need to do something, but that is a fantastic challenge. Look at your major encyclopedias, they have a team of several thousand to do fact checking on a paid basis. I'm not saying people wouldn't fact check, but its a great challenge. How would you know that people aren't just saying its legit or not just for fun?

http://religiousfreaks.com/

Re:Easier said than done?

[truthsearch]

With the method recommended in the article the system automatically assumes that a section of text which has not been modified over a series of edits to be more likely to be accurate. It's not that someone denotes a section as fact checked. But if a page has been edited many times, yet one section of it has not been modified, it assumes that unmodified section is more likely correct and colors it appropriately.

Re:Easier said than done?

[beheaderaswp]

Well the biggest problem they are dealing with is standards. In academia there's a criteria for what constitutes a "fact" and a formal process for fact checking. On the "internets", well anyone can submit edits, and claim a factual basis for it.

So a reputation system is pretty useless because special interest groups can mobilize to skew reputation.

Want to have Intelligent Design show more favorably? Ok, get a bunch of like minded people to raise your reputation.

Heck, we even see it on Slashdot when a conservative or liberal viewpoint gets buried in moderation because people of a certain political belief gang up on the opposition.

A lot of what Wikipedia is dealing with is a direct result of the deep divisions on our society. And the fact that unlike World Book Encyclopedia, apparently *everyone* is allowed into the research offices. In a virtual sense of course...

Re:Easier said than done?

[miyako]

There are two big problems I see with this:
The first is that there are a lot of articles and sections of articles in wikipedia that are heavily edited without the facts changing much. This is mostly a good thing, cleaning up grammar, etc. But the if that is used as a basis for how reliable the information is, it could be misleading because the software won't know if the facts have changed, or just their wording
The other problem that I see with this is that it makes it easy for people who "disagree" with facts to make edits to the sections to reduce their rating without just deleting them. It just makes me thing of those people who say "yeah, but evolution is only a theory" to undermine it, I can see them making minor changes to wordings of things to make the facts seem less debatable.
Of course, if someone was doing that, it would be impossible to say if they were doing it because they wanted to supress facts by making them look less reliable, or if there were simply trying to contribute to the quality of an article.

Re:Easier said than done?

[Anonymous Coward]

For instance, something has been wrong with an article from the very begnining http://en.wikipedia.org/w/index.php?title=Gato_cla ss_submarine&diff=2048162&oldid=2048151 Like the Horsepower on the submarine being four times its actual size. Then it gets repeated on every single entry for the class http://en.wikipedia.org/wiki/USS_Tinosa_(SS-283) and then gets repeated as fact on thousands of wiki-replicas http://www.google.com/search?q=four+6500-hp+Diesel +engines to the point where you can only find the truth on a few sites obscured sites if you search for the HP of a Gato class submarine. http://www.bergall.org/mechinfo.html It doenst matter how long it has stood, and especialy not that other sites share the same misinformation, all that matters is if someone knows what they are talking about, or if someone's "source" got their information from wikipedia to begin with.

I can give you the answer without even RTFA

[RLiegh]

That answer is "no". We've seen numerous ratings and karma systems set up on a variety of boards and time and time again they've been defeated by people willing to take the time to game them for whatever reason.

It's typical nerd hubris to believe that you can solve social problems through technological means.
It's been proven time and time again that you can't.

Re:I can give you the answer without even RTFA

[Anonymous Coward]

A rating system is a social solution to a social problem.

Re:I can give you the answer without even RTFA

[NewWorldDan]

Firstly, the word was "improve", not "solve". I think Wikipedia would improve substantially if it added an editorial supervision system. For example, changes were not posted until approved by a randomly assigned editor. The random part is important. Sure, it's still possible to trash the system, but that takes a lot more effort. And then you need a rating system for editors, and so on and so forth. But the Wikipedia is run on a volunteer basis. There are limits to what it can accomplish without resorting to professional oversight, which would change the very nature of the beast. Ultimately, I think we just have to accept that it is what it is.

Re:I can give you the answer without even RTFA

[Somatic]

I was tempted to mod that down cause it would have been funny. But no one but me would have got it.

Re:I can give you the answer without even RTFA

[cavintage]

There are successful reputation systems out there. For instance, Credence can avoid Gnutella spam. They have a cool algo for detecting when a group of fake users all rate the same bogus files up. Wish this sort of thing was more widely deployed. Bitzi is the dumb version of the same idea, but never worked for me.

Re:I can give you the answer without even RTFA

[catbutt]

Good the Wright brothers didn't say that because lots of attempts were made at flying that failed.

Slashdot's karma system is far from perfect, but at the end of the day it works. Can you game it? I don't really think so, at least not without a LOT of effort, which generally means contributing a lot of good content/ratings so that you can sneak in a very small amount of biased content or ratings.

Whether "ungameable" is possible or not I don't know, but I am quite sure that wikipedia's system could be improved upon massively.

Re:I can give you the answer without even RTFA

[AndyG314]

I think that almost any reputation system woulth threaten the impartial nature of wikipedia.

Re:I can give you the answer without even RTFA

[Yokaze]

> It's typical nerd hubris to believe that you can solve social problems through technological means.

That is not what nerds are trying, it is what society is doing: Trying to solve social problems through software for wetware (laws).

In the case of computer based communities, that laws are codified in programming languages, whereas in RL it is codified in legalese.

> It's been proven time and time again that you can't.

Yes... like flying.

I admit, my first statement seem to be more an argument against the possability of creating a good working online community, as we cannot really claim to have solved our social problems in real life.

However, one has to remember, that an on-line community only has to solve a small subset of the problems, which one has to solve in real life.

IRC, some time ago (one or two years ago) there was a post on Slashdot about a sociology study on the matter on reward and penalty system in communities, which claimed to have isolated some simple rules for a thriving one. I cannot remember having seen it implemented in software.

Re:I can give you the answer without even RTFA

[Anonymous Coward]

Slashdot's karma system works fine for Slashdot, but wouldn't work at all for Wikipedia. Slashdot, despite how much you may like it, is pretty far from neutral. It does the same thing this reputation thing would, represent the opinions of those most active in using it. Wikipedia would be ruined.

Re:I can give you the answer without even RTFA

[catbutt]

How does wikipedia, in its current form, *not* represent the opinions of those who use it (including the opinion that it shoudl be as neutral as possible)?

If people using it today value neutrality, how is that going to change when a karma system is put in place?

Re:I can give you the answer without even RTFA

[Jerf]

But he's not talking about assigning reputation to people, he's talking about assigning reputation to content.

There is a world of difference.

Re:I can give you the answer without even RTFA

[geekoid]

Wow, worst analogy on slashdot ever.

The wright brothers weren't trying to overcome social issues, they were overcoming a technical issue.

You can not solve social issues with technology. Social issue aren't always rational.

I have gamed slashdot karma many times.
In days of yor, I could intentional get a 50 karma in a week, and a -49 karma the next.
I can do it today.

Don't believe me? mod me up. ;)

Re:I can give you the answer without even RTFA

[mdwh2]

Slashdot's karma system is far from perfect, but at the end of the day it works.

I disagree. I set my preferences so anons don't have a -1, and people with karma bonuses aren't shown with +1.

If you have something good to say, it should stand on its own merit - and that applies to Wikipedia. We should accept information there because it is verifiable, and not because we trust the random person who wrote it.

Re:I can give you the answer without even RTFA

[mdwh2]

How does karma stop spam? Not moderation (which Wikipedia already has on a far more powerful level than Slashdot, in that edits can be reverted rather than simply moderated down), but the karma bonus system? I see no way to distinguish the trolls from the majority of Slashdotters who are not trolls, but simply do not have a karma bonus.

Re:I can give you the answer without even RTFA

[jZnat]

Then it isn't a wiki anymore. I'm sure a lot of edits these days are clarifications, typo fixes, and other minor modifications. Having to approve them all would be tedious and would grind Wikipedia to a halt.

The Slashdot moderation system proves....

[no_nicks_available]

this wouldn't work.

Re:The Slashdot moderation system proves....

[joe 155]

I must disagree, the /. system is actually working pretty well. If you say something which is needlessly offensive you will be modded flaimbait, the same would go if you're trying to start a flame war with comments like "GNOME smells of cheese and suX!!11!". If you make some "GNAA!!!!!" type posts, that'll be a troll. If you say something which is completely off topic, it gets modded as such. Both of these things mean that modding becomes pretty much a true/false kind of thing, which meta modding can comfirm. It also stops being seeing it as a default and makes the best shine out...

Possitive modding is a little more shakey with "informative/interesting/insightful" all meaning pretty much the same thing in most people's mind, but that's not too much of a problem.

Group think can cause issues, but in reality there is such a wide range of modders it is often avoided (you can see some pro-MS or anti-Apple comments come through)... although the system isn't perfect I guess group think at least only makes content that most would want to see if they come here.

It is also interesting to note that most people do care about karma and do like to get modded +5, maybe the wiki system would work in a similar way - where people will care.

Re:The Slashdot moderation system proves....

[AlphaWolf_HK]

If you say something which is needlessly offensive you will be modded flaimbait, the same would go if you're trying to start a flame war with comments like "GNOME smells of cheese and suX!!11!". If you make some "GNAA!!!!!" type posts, that'll be a troll. If you say something which is completely off topic, it gets modded as such. Both of these things mean that modding becomes pretty much a true/false kind of thing, which meta modding can comfirm. It also stops being seeing it as a default and makes the best shine out...

This is not true. I have seen posts where somebody points out a mere fact about microsoft that gives them a positive light get moderated as a troll just because somebody doesn't like microsoft. It also occurs often that somebody will post comments that have mere facts which support a particular conservative viewpoint, yet just because somebody who may be a liberal doesn't like it they downmoderate it as overrated or as flamebait.

Then the karma system basically says "don't make these kinds of comments again, or else all future comments you make will be ignored." People realize this, thus they are reluctant to speak their mind when they already know that the group (due to its demographic) is going to disagree with them. By having a karma system, you essentially introduce the "fear" element common among e.g. dictatorships. That is, fear of speaking your mind lest you offend somebody who reduces your group reputation. Another fitting way of describing group think would be "group censorship."

The meta moderation system attempts to solve this, but it falls far short for various reasons. First of all the meta moderator could agree with the moderator, even though both of them are biased. Second of all, not everybody wants to spend enough time reading each subject and then each post in order to understand the context fully, but they still want the moderator points anyways so they may just pick answers at random.

This would be terrible for an encyclopedia, since if somebody doesn't like a fact (even though it may be true) they can go along with the groupthink and censor it anyways. An encyclopedia is not, nor should it ever be, a democracy. An encyclopedia should follow the facts as they are, not the facts how people want them to be.

If you still feel that this isn't the case, then explain why it often occurs that people feel the need to post as anonymous coward when posting a view that they already know the group won't like?

Re:The Slashdot moderation system proves....

[mdwh2]

And what is wikipedia today if not "groupthink"? As much as they like to say their is "NPOV" and that this is purely objective, I call BS.

But if what you say is true, then a karma system would only make this worse. Who would get the most karma? Those people who are spending all their time editing on Wikipedia, of course.

rep farming

[Sebastopol]

of course it won't help. people will just grind for rep and then vandalize.

what we need are national ids and biometric logins.

i kid... i kid...

Re:rep farming

[jelle]

"of course it won't help. people will just grind for rep and then vandalize"

It's a form of 'payment'. Need to 'do good' before can 'do bad'. How wouldn't that help if vandals need to make (e.g.) 10 useful contributions for each vandalization instead of just vandalizing like it's possible now?

With the example above, for every 10 vandals, there only needs to be one more vandal still building up his reputation to undo the vandalism of all the others...

And that's without the people who don't vandalize.

Re:rep farming

[Sebastopol]

The problem is the bandwidth and debate required to decide what is a USEFUL mod and what isn't. By inserting a person or people into the loop you do two things: 1) bottleneck wikipedia immensely, and 2) require a subjective analysis of each edit by an inherently biased 3rd (er, 2nd) party.

* Yes, there's the karma-like concept, but that could easily be spammed by the opposition, just like it is on /. with alternate login IDs. Metamoderation taught me moderation is probably only (just a guess) 50% effective because lots of valid points end up as trolls, crap gets modded as insightful, and over time people move on, leaving bad mods.

solution in search of a problem

[capoccia]

this is a solution in search of a problem. wikipedia does not have a problem with ordinary vandalism that could result in a reasonable measure of a user's reliability. wikipedia's biggest problem is with unfounded but believable information. in this case, the measure of reliability of a user would be nearly useless because the reliability of their edits is unknown.

Re:solution in search of a problem

[owlnation]

Yes, absolutely, I couldn't agree more.

I am progressively more and more disturbed by the wikipedophile focus on finding a solution to "vandalism". Liken it if you will to an right wing politicans campaign to rid the world of "terrorism".

I would concur that mindless destruction of someone's work is annoying and should be dealt with, however, I think it is important to understand that not all acts of destruction are in fact mindless - some are legitimate protests. Much as wikipedia likes to harp on about vandalism, I don't yet see a valid definition of what constitutes vandalism.

Also, what I feel is potentially the most dangerous aspect of Wikipedia is the ability of a moderator to deliberately manipulate information to promote agendae. Wiki's editors are self-appointed and not accountable to anyone. For as long as wikipedia remains in the form it is in; by small changes, small spins and offering seemingly plausible facts you could potentially alter the course of history. The ability to alter the perception of history and facts is right there on a wikipedia page. I am astounded that people are not already doing this - I'm sure they must be. It is incredible power.

While the mindless vandalism is annoying, removing moderation and removing protection is much safer in the long run. Since, in this way, people will know not to take Wikipedia seriously as the single source of any fact, and also because it will be much harder for extremists to push agendas. We as a community can contribute to keeping the site tidy, rather than have a secret police force doing it for us.

Quis custodiet ipsos custodes? We actually need an answer to this question before wikipedia does anything else.

How about one for /.?

[ackthpt]

I mean, we could all moderate/evaluate the slashdot editors on their choice of stories and keep stats, like onna baseball card.

CmdrTaco
Dupes: 23
Veiled ads as news: 18
Old news: 17
Allowed Bad Grammar: 2,980
Allowed Bad Spelling: 9,874,376

Yes

[neonprimetime]

Could a Reputation System Improve Wikipedia?

YES - It works on /.

Re:Yes

[treeves]

Yeah, new improved: with Wiki-karma whores!

Re:Yes

[drinkypoo]

While the slashdot moderation system is complete crap, can you imagine what this place would be like without anything at all? Me neither.

What about...

[MarcoAtWork]

The site is /.-ed, but this got me thinking: what about having an additional page view that uses color to highlight text age? Oldest text would be black, newest would be something else (red? blue?), intermediate 'ages' in intermediate shades. This would make it quite obvious which parts of the article haven't been modified in a long time.

Re:What about...

[tonyr1988]

Better yet, we could use MySpace color schemes.

• Oldest: black with jet black background
• Little newer: Yellow on orange
• Tad bit newer: Light blue with seventeen exclamation points
• Couple days old: ALL CAPS BABY
• Posted today: Embedded into a crappy Flash movie from www.pimpandrockoutyourwikipediavids.com

Then it will be truly distinguishable.

Not a complete solution

[br00tus]

On Wikipedia, pages relating to quantum mechanics are sometimes vandalized, but 99% of people are on the same side in terms of keeping it accurate. So there are various ways this can be improved.

On the other hand, the pages regarding the fight between Hamas and IDF are as much a battleground as is the area around the Israeli/Lebanese border. I have been involved in Wikipedia for years and have just seen things deteriorate around these types of flame-wars. Wikipedia's leadership is not dealing with it well. Imagine Slashdot setting up a wiki where we had to determine which was better - Debian or Gentoo (or Ubuntu etc.), BSD or Linux, vi or emacs etc.

We are technical people, and there's the old thing about when you have a hammer everything looks like a nail. But I don't think a technical solution will help much in regards to this. I'm not even sure you really can have a neutral view about wars in the Middle East. And even if you could, Wikipedia's "cabal" is nowhere near able to deal with it, and I doubt they ever will be. Personally, I think most of the people in high positions at Wikipedia are jerks, all the flamewars and such seem to have driven most of the nice people off.

Things like Wikipediareview.com convince me that what will ultimately happen is alternatives to Wikipedia will pop up. Wikipedia is a new phenomenom, and it makes sense everyone edits on the same wiki, but why should that be? Why should pro-Hamas and pro-Israel people edit and battle on the same wiki? It makes little sense, and I'm sure in time, just as IRC went from one network to EFnet and Anet, and then split even more, I'm sure we'll see splits with Wikipedia. In the old days, the Encyclopaedia Britannica had one view of history and the Great Soviet Encyclopedia had another, why should the future be any different?

Re:Not a complete solution

[Kesch]

On Wikipedia, pages relating to quantum mechanics are sometimes vandalized.

Actually, every article on quantum mechanics exists in a state between vandalized and not vandalized. By viewing it you colapse the waveform and change it's value. Now, there is a good probability that it will turn out unvandalized, but as you have stated it occasionaly collapses into a vandalized article. After you leave the page, Wikipedia runs complex calculations in it's improbability engine and sends the article back into a quantum state.

P.S. This is presented as per my understanding of quantum mechanics which I learned entirely from Wikipedia. It may be wrong however as my viewing might have caused it to appear in a vandalized state.

P.P.S

Imagine Slashdot setting up a wiki where we had to determine which was better - Debian or Gentoo (or Ubuntu etc.), BSD or Linux, vi or emacs

Debian, Linux, emacs. That wasn't so hard. (Anyone who disagrees is a terrorist.)

Re:Not a complete solution

[Anonymous Coward]

I'm not even sure you really can have a neutral view about wars in the Middle East. ... Why should pro-Hamas and pro-Israel people edit and battle on the same wiki?

Because the harder you try, the closer you come to success.

In contrast to most articles on the Israeli-Palestinian situation where the writer is trying to advance a particular viewpoint, I've actually found the Wikipedia articles to be quite good. You actually have a situation where people on both sides of the issue are coming together to try to figure out what the facts are. In my experience, it's the extremists committed to propagating factual inaccuracies that eventually get driven away from wikipedia because, unlike when they preach to their choirs, the audience at wikipedia actually cares whether the articles are factually accurate.

On the other hand, I have found some of the high traffic articles to be overly neutralized. For example, I was interested in the crimminal justice system in Singapore but, when I went to the main article on Singapore, all it contained was bland touristy propaganda. One of the main things Singapore is known for in the world is harsh punishment of minor crimes but there was only one sentence about it buried in an unrelated paragraph way down the page.

Vanadlism is not Wikipedia's main problem

[gnetwerker]

In my opinion, vandalism is not the primary Wikipedia problem. Yes, it is embarrassing, but ultimately only a secondary symptom of the central problem: when you have an "encyclopedia that anyone can edit", anyone does edit it. The clear observtion (can't remember who said this first) is that twenty teenage idiots do not collaboratively make an expert. The perhaps more important corollary is that twenty teenage idiots plus one expert are indistinguishable from twenty-one idiots.

Larry Sanger has acutely commented on Wikipedia's anti-elitism and the way they have run experts off the system. Experts don't have the time or energy to debate fundamental points of well-understood scholarships with game-playing trolls. Further, even when they aren't teenagers, Wikipedia has become the home of everyone who wants history and scholarship to read the way they like it rather than representing some academic consensus. As a result we have politicians trying to rewrite their personal biographies (or those of their opponents), partisans on each side of the world's conflicts burnishing their allies and undermining their opponents (Israel/Palestine, Turkey/Armenia, US/everyone else), and devotees of everything from Microsoft Vista to Nintendo to PETA skillfully expunging objective truth from their deifications of the chosen object of worship.

So doling out karma to 100,000 teenage idiots is not going to solve Wikipedia's problem. In order to save Wikipedia, we need to destroy it -- it needs to be edited by more experts and fewer "normal people".

Re:Vanadlism is not Wikipedia's main problem

[Anonymous Coward]

Guess what? Anyone who thinks that Wikipedia some sort of expert-system and that it is being dragged down with anti-elitism has no idea at all what WP has become.

Right now, Wikipedia strives to attribute all statements in its articles to a reliable source. This is something an idiot can do, once they are shown how.

Assuming that debates would somehow be fewer or settled faster and with universal agreement if there were more 'experts' shows no understand of human nature what-so-ever. The best WP can hope for is that everyone agrees that, yes, a prominent NYC newspaper says {Hezbollah statement 1} and a prominent Delhi newspaper says {Hezbollah statement 2}. Note, this results in information that is usable.

If an expert-based WP would work, there would be one. At best, it would grow slowly and many topics would be half-complete. Think of the times when you've looked for an obscure topic on the web, only to find that the best source of information is someone's personal web project that was last updated in July 2003 with "Sorry about that long absence, I'll update more from now on." If you insist on having an experts-only volunteer system, that is what you should expect.

If you're happy with the goal of having all important (or sometimes just -all-) facts related to a topic located on one mostly up-to-date page arranged in a somewhat-organized manner, then join me as a cautious WP fan and contributor.

If you insist on anti-anti-elitism, then I suggest you get all of your information exclusively from the most expensive peer-reviewed journals possible, but make sure that you personally investigate the backgrounds of every author and duplicate experiments when possible, because nobody in the world but you can do it right...right?

If you're not happy with this false dichotomy, then note that I only cared enough about what you have to say to write this post, and not a damn cent more.

One Idea from the artical that was worth while

[AndyG314]

First off, most of this artical was a bad idea. One thing that did seem like a good idea, was to somehow (perhaps by marking in red or some other visual clue) indicate what part of an artical was new, from the part of the artical which has existed for a while. This would help in several ways:
1) People looking for reliable information would know that these parts of an artical have not been exposed to long term scrutney, and therefore may be inacurate.
2) The new, and therefore unverified parts would be more obvious, which would help focus accuracy checking on new material.

It would seem logical for "new" text to remain new, untill it had been viewed a certain number of times, allowing enough sets of eyes to read it, rather than a set time limit, since some articals are not viewed very often, which allows them to remain inaccurate for a long time.

I dunno...

[earthbound kid]

According to some preliminary research by Aaron Swartz about who write Wikipedia, while it's true that most of the editing is done by regulars of the sort who would have karma, most of the original content is added by people with few other contributions to Wikipedia. The regulars just go back and put everything into Wiki format, add tags, make things follow style guides, etc. Since the real work is done by anonymous people who may never come back to the site, it's important to keep the process as open as possible for people who are still new to Wikipedia.

Re:Hardly

[catbutt]

Yeah but some of the most brilliant things on the internet might have a low "page rank" on Google. Still, Google's reputation system (which is exactly what it is), does a pretty good job, even with the fact that it must infer ratings by links. Obviously, they have to work pretty hard to make it hard to game.

So now we rate people too... Bad move.

[kinglink]

I'm not going to get in the politicing and all. The simple fact is the only response back you'll get from this is how many reverts have been done when you post and those arn't always your fault.

The best parts of Wikipedia is a fast and easy way to edit information, no hassles, no extra effort required. You get out what you put in and that's it. You want to put in the work to be a vandal you're a vandal, but in the end you already know what you're doing. Type in a good sentance but someone replaces it with a better paragraph that's fine.

But instead of working on the core of the experience now we are going to spend time rating each others' facts, rating each other. Basically just killing time. The simple fact is we don't need it, this system is in place in a lot of other places and in effect it basically weeds out the bad apples at the inconvience of all the good users. "You'll have to do 5 discussion posts before you can edit an article" "you have to edit three more articles before you can add an article". This stuff doesn't help or appeal to anyone but "karma whore" types.

If I write a well written page about the new player on the Red Soxes, I should be able to go in to a page that links to it create that page, set up my links and go. I should be able to do this on the first day as well as the fifth year with the same ease. Adding in safe blocks and guards will only hurt wikipedia's overall goals, not help the ideas it promotes. The best thing to do is start handing out serious penalties for vandalism or obvious weasel words.

This doesn't even get into the idea of being able to do fast edits with out logging in, something that's helpful at times.

Re:Hardly

[Millenniumman]

The editors have a conservative streak? That is amusing. Or are you forgetting the editors' comments (in the story summaries) about the free market being a failure, fahrenheit 9/11 being very insightful and wishing it would sway voters, and numerous other such comments? Most of /. is very far from conservative.

technology is the answer

[klenwell]

One point with Wikipedia that seems to get overlooked -- or at least taken for granted -- is the power and ingenuity of the code that runs it. Technology is part of the solution here. If nothing else, Wikimedia deserves credit for putting together a state-of-the-art wiki machine -- an open source state-of-the-art wiki machine. Some of its features are dauntingly obscure and complex but it falls back quite gracefully to allow even the newest user to function with it effectively. I'd argue Wikipedia has succeeded in large part due to the technology.

That said, there seems to be two alternate proposals here in the summary. (1) A karma system and (2) a new color-coded visual feature. I agree that #1 would be vulnerable to all sorts of gaming schemes -- which isn't to say it wouldn't help, but it'd have to be smart. #2 sounds like it would be a more unequivocal benefit.

Both would be interesting innovations and consistent with the progressive user-friendly code behind Wikimedia.

Web of credibility

[jemenake]

About 10 years ago, when I learned of PGP's "Web of Trust" system (where I could choose to trust everybody that you trust), I turned to a colleague and said "What we need next is a 'Web of Credibility'..." where individuals in the community can bestow credibility points to others... and the points I can bestow to you would depend upon how credible the community thinks *I* am, and so on. In other words, if Noam Chomsky or Lester Thurow vouch that you're highly-credible, then that would boost your credibility more than a few glowing scores from your cable guy and the kid at the local sip-n-go. Ultimately, it would be a measure of how likely (or unlikely) you were to spout off on something that you had no clue about.

Having not yet RTFA, I'd just like to say that I agree, wholeheartedly, with the general notion... and I look forward to the day when our credibilities are incoporated into our digital signatures (that I hope we're also all using someday). - Joe

Re:This is bullshit...

[KillerBob]

...and I'm not even speaking about the validity and effectiveness of a karma point system. I mean, a visual queue to tell people what content to believe or not? What happened to reasoning, critical thinking and the scientific process? Do we need to think for ourselves or rely on someone's visually appealing color code to know what or what not to trust?

Sometimes it's obvious when a factual error has been made. Say, for example, somebody changes the article on Monarch butterflies to claim that they feed on the blood of human babies. Anybody with an ounce of reason can see that it's a fallacious claim. But what if it's a minor factual error that can easily slip past your notice? Or worse... an outright lie that seems more reasonable than the truth? Say, for example, somebody claims that the volume of a mol of helium is 23.6L. Outside of people with an actual background and education in chemistry, nobody's going to notice that error. Critical thought or no, that's something you just have to know in order to see as wrong. Those with a background in Chemistry know that it's actually 22.4L. Those without have no clue.

And the problem arises when people then use that number as fact, without bothering to do research outside of Wikipedia. A *lot* of research papers, particularly in High School and Undergraduate studies, skimp on research. I know people who have never been to a library to research, and do all of it online. These are the people getting burned. Arguments over the morality or wisdom in doing all of your research in that manner aside, Wiki is fast becoming the prime source of research data. And that's why the people at Wiki are trying to find a way to improve their credibility.

Yes, but worth the effort

[RareButSeriousSideEf]

Pardon my nerdocratic hubris here, but IMO Wikipedia would be a fantastic petri dish for evolving a robust reputation system, and the result could be useful in a wide variety of applications that facilitate pseudonymous communication & transactions.

In the beginning, I'm sure this would just gather data & have little to no impact on the content. But over time, it could well become increasingly effective at improving content quality as its designers started to identify patterns & meaningful correlations in the collected data.

This isn't so different from SPAM filters that need constant training, or PageRank, or eBay feedback scores, or AVN forum posting rules, etc. One needn't restrict the reputation data to any one data species; you could use a composite of community feedback + usage statistics + genetic algorithms etc., and over time tweak the weight any category of data is given to account for its sample size, its expected margin of error, and its track record in terms of predictive power.

Sure, it's a time consuming undertaking & it'll take patience before we see results, but I don't see the real difficulty being in rigging up the system; I think the real difficulty will be in defining exactly what constitutes a quality article.

Now, take a minute to share a utopian dream with me: Imagine the day when registered Wikipedia users with good reputations will be able to make edits from a Tor connection. :-)