Wikipedia and the Politics of Verification

Slashdot regular contributor Bennett Haselton writes "The reports of Sinbad's death become greatly exaggerated. A Wikipedia contributor is unmasked as a fraud, raising questions about why he wasn't called out earlier. NBC airs a piece about how anybody can edit any article on Wikipedia, and errors creep in as a result. (Duh.) But what's most frustrating about all these controversies surrounding Wikipedia is that news reports describe these incidents as if they are a permanent, unsolvable problem with any type of community-built encyclopedia, when in fact there seems to be a straightforward solution." More words follow. Just click the link.

In its simplest form, couldn't a person's academic credentials be verified by sending a confirmation link to their .edu e-mail? (Which could be identified as a faculty address either by a domain name like "faculty.schoolname.edu", or by a Web page in the faculty section of the school's Web site identifying the person's e-mail address?) And then once the user's bona fides have been verified in this or some other way, couldn't they put their seal of approval on any article whose contents need to be considered reliable, or that readers want to cite as an authoritative source? In this way, with only a few minutes of effort and without changing a single word of the article, its value is increased many times -- surely one of the best possible trade-offs in terms of effort versus reward. (As for the question of "What experts would do this?", the answer is, presumably the same people who contribute to sites like Wikipedia currently. If their motives are altruistic in the first place, hopefully they would be willing to take this extra step if they knew it would increase the article's usefulness.)

Something like this model is planned by the operators of Citizendium.org, a Wikipedia alternative (I balk at using the word "rival" although it is inevitable that people will see them that way). The last time I wrote about Citizendium, some thought it sounded like such a valentine to the project that they wondered if I was a shill; actually, sometimes a project just comes along that aligns almost exactly with what I would have done if I could have re-done a popular project like Wikipedia with a few design changes, and when that happens, I just say so. Some others may have wondered if I was sucking up for a board position or something. No, that would be, like, work. But I think they have some good ideas that will make them a more useful alternative in some cases, unless Wikipedia copies back some of their ideas in order to serve both needs at once, which would also be a good thing.

Consider the two major issues on which Citizendium is planning to take a different approach from Wikipedia: (1) user verification, and (2) putting published articles into an "approved" state under the stewardship of a credentialed editor, who has to sign off on any future changes to the article. The issue of user verification can be further divided into two sub-issues: (a) verifying users for the purpose of ascertaining their credentials, and (b) verifying users for the purpose of limiting the amount of vandalism committed by new users under pseudonyms. (While editorial control on Citizendium means that it is not possible to vandalize the public-facing version of an article after it has gone into an "approved" state, users can still vandalize an article while it is a "work in progress" being built up towards the first milestone where it can be approved. Citizendium founder Larry Sanger says that such vandals are surprisingly, pathetically motivated even though their work is only seen by a small audience.)

On the first issue, the one of verifying user credentials, I think the verification of .edu addresses especially would be a cheap and easy way to increase the value of every article that that user writes, or signs off on. I don't think, however, it's necessary to go as far as Citizendium is currently planning on going, by requiring real names and biographies of all users. My thinking is that if an article is synthesized by 100 monkeys with typewriters but the finished product is giving the blessing of a credentialed professor of physics, it's pretty much just as reliable as if the professor had written it themselves. And if the same article gets the blessing of multiple credentialed experts, it could justifiably be considered more reliable than many printed sources written by a single author. The point is that the credentials that matter, are those of the people who stake their reputation on the accuracy of the article, not necessarily those of the people who contribute to it. So on this front, I think that while Wikipedia asks too little of users' backgrounds, Citizendium's current plan would ask too much, because as long as you have the credentials of one person who has signed off on an article, collecting non-verifiable bios of the article's other contributors doesn't actually gain anything.

The other side of verifying credentials is the use of credentials to prevent vandalism. In this situation it's not necessary to verify that the user actually is who they say they are; the system only needs to ensure that the same user is not signing up over and over again after previous accounts get banned for abuse. (You could ban users by IP address, but tools like Tor make it easy for users to connect from what appears to be a different IP address every time.) A blog post from Citizendium founder Larry Sanger lists three possible approaches instead: (a) requiring existing user X to vouch for new user Z before Z can join; (b) requiring new user Z to provide a link to a "credible" Web page establishing their identity; or (c) requiring new user Z to provide a link to a "credible" Web page of some person X who can vouch for Z's identity. I don't know how quickly a system could grow by referrals only -- after all, I was surprised that GMail took off so quickly during the period when you could only join with an "invite" from an existing user. Then again, GMail was giving away something for free that almost everyone could use, so most people who wanted it, would find themselves closely linked to someone else who had it. Citizendium, on the other hand, asks not what they can do for you but what you can do for them, and so might not achieve enough penetration to spread by referrals only.

I suggested that one alternative would be to send a postcard to each new user's physical address with a unique six-digit number, which they would have to enter in order to complete their registration, in order to verify that new users really were unique. The problem here, apart from the privacy concerns, is the delay that users would incur before their registration was complete, which would take away the "instant gratification" that they could get from starting to contribute right away. (You could let users edit before their address is verified, but that would just enable the same person to keep re-creating new accounts with unique but fake addresses, and use them to commit vandalism before the account was found out.)

Another idea would be that for new users, their first, say, three edits would go into a queue to be reviewed by verified users, and once the first three edits have been approved, the user is able to make edits in real time. (Since anybody would be able to review a new user's edits to make sure they were not spam, the new user's edits could be reviewed very quickly, since any Citizendium volunteer who was online, could review the latest entries in the edit queue and approve them.) It's true that a user could game this system by, for example, submitting three minor improvements, and then using their unblocked account to vandalize articles while they're being worked on. However, even in this case, the "vandal" would probably end up having a positive contribution to the site, because of the three small improvements that they'd already made. If a legitimate Citizendium volunteer would have to spend more effort making those three small improvements, than it would take to let a new user make those constructive changes and then ban them and revert their destructive changes once the user is caught committing vandalism (and the latter wouldn't take much effort at all), then Citizendium has actually gotten a good deal out of the "vandal"! (To make this work, a user's first contributions could not be "neutral" changes like replacing one word with a synonym; they would have to be actual improvements, even small ones, thus ensuring that the net effect of a potential "vandal" is positive.) There may be other possible solutions. These are just alternatives in case the model of referral by trusted users turns out not to work.

Now switching to the other side of the reliability issue: Whether the default article that is displayed to the public for a given topic, should be the latest "stable" version approved by credentialed users, or the very latest version incorporating all edits submitted by any user whatsoever. Having talked with members of the Citizendium and Wikipedia communities in their respective forums, there appear to be three schools of thought on the article stability issue. The first is that the whole idea of putting articles into an "approved" state and moderating all changes going forward, goes against the "spirit" of wikis in general and Wikipedia in particular. The second, suggested on the Wikipedia discussion list by Sheldon Rampton, is that it would be a useful feature if credentialed users could select certain page versions in the page history and "sign off" on the accuracy of one of those past versions; the page displayed by default would be the bleeding-edge latest one (with all of the possible vandalism and inaccuracies that entails), but users who wanted a reliable, citable source could look in the history. The third school of thought is that reliability is so valuable, that the default page displayed to the public and carrying the stamp of the project, should be the latest version approved by credentialed editors -- the model that Citizendium currently has in mind.

I'm not really partial to the first view, since I think the success of the project should be defined by how it achieves its goals (whatever you define those goals to be) and not in whether it kept with its original "spirit". Since Wikipedia has far more readers than contributors, if your motivations for contributing to or maintaining Wikipedia are at all oriented towards doing good for other people, presumably meeting the needs of readers is more important than keeping the party going for contributors (provided, of course, that the environment for contributors is at least pleasant enough to keep them contributing). The choice between the second and third points of view is more interesting. There's no obvious best-of-both-worlds choice here, because what motivates many contributors (the fact that their changes go live to the entire world, right away) is also what motivates vandals.

On the other hand, the problem doesn't sound unsolvable. You could go with the Citizendium model of editor-approved changes but create a prioritized system for "urgent" updates, in the case of changes to an article made to incorporate current events. Suppose users (who have been verified using one or more of the methods above) are each issued a certain number of "credits" that they can use to mark a proposed update as an urgent, breaking change. (Misusing these credits to mark changes as "urgent", that really aren't, would be considered abuse tantamount to spamming or vandalism.) Then let's say, for example, Anna Nicole Smith dies. A user could submit this change to the Anna Nicole Smith article, along with a link to a reliable news source (e.g. a wire service story) and a credit marking the change as "urgent". Since an editor would not need any particular expertise to view the article and verify that the change was accurate, any editor could review the "urgent request queue" and approve that particular change for publication, ensuring that the queue was checked frequently throughout the day and urgent updates would get pushed through quickly. Thus the site could keep pace with breaking current events without the kind of inaccuracies that plagued Kenneth Lay's Wikipedia entry when he died.

So there's a trade-off there, between displaying all the latest changes by default and motivating people to contribute but also running the risk of vandalism, versus displaying only the latest editor-approved page. Where there is not a trade-off, that I can see, is in the option of simply having an editor-approved version of a given page -- whether it's displayed by default, or only stored in the version history where people can look for it. To me, both of these steps seem to consist of pure gain for relatively little effort:

1. Verify credentials of academic professionals by poking their .edu address.
2. Allow them to give their "blessing" to certain versions of a page in the page history, so that users can rely on those specific page versions and even cite them as sources where appropriate.

So I hope that Citizendium will help bring more prominence to the idea, and that something similar might get incorporated back into Wikipedia. The approval of an identity-verified expert can improve an article's value so much, for such comparitively little extra effort, that it makes no sense not to have that option.

Comments

Re:Even simpler way to verify if someone is a teac

[Bloke down the pub]

I'd say 99.9% of University's have the number for their switchboard on their website

More likely scenario:

You: Can I speak to Professor [Insert Name Here].
Operator: He's very busy, so first you have to prove you're not just some time-wasting 'tard. So, what's the correct usage of an apostrophe?
You: To ... signify a plural.
Operator: You fail it!
You: Ummm, to warn that an "s" is coming?
*clunk*

Stephen Colbert

[just_another_sean]

Nothing to say really, it just didn't seem like a complete discussion of Wikipedia without a mention of Mr. Colbert.

The poor remain poor because of their own choices.

[NDPTAL85]

"You know why I don't like the way the market works? Because rich people fuck with the market and make it work in their favor. I wouldn't mind so much if people who were rich actually deserved it because they were smarter or worked harder, but the majority of rich people aren't and don't. They are rich because money breeds money, and because they have no compunctions about fucking over others."

Is it really this way though? If it is then you're right and it sucks. But what about personal responsibility? When people, poor and middle class, constantly make bad choices and decisions that keep them poor how is that a rich person's fault? Why do you automatically assume the world is the way it is because of the plans of the rich instead of the faults of the poor?

"Race is just a red herring, though, thrown out by the rich to keep the rest of us fighting amongst ourselves instead of uniting to fight them. There are far more middle class and poor people than there are rich people, and if we want to we can just take all their shit and there is nothing they can do. Yeah, that's right you rich motherfuckers, the police and military are all poor and middle class to, who do you really think they'd side with?"

Well seeing as that hasn't happened I'm guessing the police and military would side with the side of law and order. While the police and military are mostly made up of poor and middle class families they do have HOPES and DREAMS of rising above that. They don't want to fight the rich because they want to BE rich themselves. And the way for them to do that is not to overthrow the status quo but to make it via their own merit. In any case its irrelevant. After EVERY revolution, EVERY SINGLE ONE, a new elite forms. You just end up shuffling the deck a bit. As time goes on those who are better at their jobs and or smarter will rise to the top and those who suck or are dumb will go back down to the bottom. We'll be right back square where we started from. The collective masses of the poor and middle class are not some mindless mob who all think alike. Setting aside the very REAL and DISTINCT differences between middle class and poor folks themselves (such as middle class folks being deathly afraid of sinking down to the "poor" level and being disgusted with the habits and the way "poor" folks live their lives, and the resentment the poor harbor towards not just the "rich" but the "middle class" for daring to have more than they do.) everyone is different, there's many different groups within this lot. Your dreams of everyone uniting together against the rich are just that, PIPE DREAMS.

So whats your plan then? Constant, never ending revolutions? In other words, do you have an alternative way of doing things that WORKS as opposed to the tired old ideologies of "blaming the man" or "stealing from the rich to give to the poor (life choice makers) communism/socialism" that flat out don't work and have been collectively shat upon and rejected by nation after nation, continent after continent except for the poor deluded fools in Latin America?

I don't think you do. I don't think you ever did.