Wikipedia and the Politics of Verification

Slashdot regular contributor Bennett Haselton writes "The reports of Sinbad's death become greatly exaggerated. A Wikipedia contributor is unmasked as a fraud, raising questions about why he wasn't called out earlier. NBC airs a piece about how anybody can edit any article on Wikipedia, and errors creep in as a result. (Duh.) But what's most frustrating about all these controversies surrounding Wikipedia is that news reports describe these incidents as if they are a permanent, unsolvable problem with any type of community-built encyclopedia, when in fact there seems to be a straightforward solution." More words follow. Just click the link.

In its simplest form, couldn't a person's academic credentials be verified by sending a confirmation link to their .edu e-mail? (Which could be identified as a faculty address either by a domain name like "faculty.schoolname.edu", or by a Web page in the faculty section of the school's Web site identifying the person's e-mail address?) And then once the user's bona fides have been verified in this or some other way, couldn't they put their seal of approval on any article whose contents need to be considered reliable, or that readers want to cite as an authoritative source? In this way, with only a few minutes of effort and without changing a single word of the article, its value is increased many times -- surely one of the best possible trade-offs in terms of effort versus reward. (As for the question of "What experts would do this?", the answer is, presumably the same people who contribute to sites like Wikipedia currently. If their motives are altruistic in the first place, hopefully they would be willing to take this extra step if they knew it would increase the article's usefulness.)

Something like this model is planned by the operators of Citizendium.org, a Wikipedia alternative (I balk at using the word "rival" although it is inevitable that people will see them that way). The last time I wrote about Citizendium, some thought it sounded like such a valentine to the project that they wondered if I was a shill; actually, sometimes a project just comes along that aligns almost exactly with what I would have done if I could have re-done a popular project like Wikipedia with a few design changes, and when that happens, I just say so. Some others may have wondered if I was sucking up for a board position or something. No, that would be, like, work. But I think they have some good ideas that will make them a more useful alternative in some cases, unless Wikipedia copies back some of their ideas in order to serve both needs at once, which would also be a good thing.

Consider the two major issues on which Citizendium is planning to take a different approach from Wikipedia: (1) user verification, and (2) putting published articles into an "approved" state under the stewardship of a credentialed editor, who has to sign off on any future changes to the article. The issue of user verification can be further divided into two sub-issues: (a) verifying users for the purpose of ascertaining their credentials, and (b) verifying users for the purpose of limiting the amount of vandalism committed by new users under pseudonyms. (While editorial control on Citizendium means that it is not possible to vandalize the public-facing version of an article after it has gone into an "approved" state, users can still vandalize an article while it is a "work in progress" being built up towards the first milestone where it can be approved. Citizendium founder Larry Sanger says that such vandals are surprisingly, pathetically motivated even though their work is only seen by a small audience.)

On the first issue, the one of verifying user credentials, I think the verification of .edu addresses especially would be a cheap and easy way to increase the value of every article that that user writes, or signs off on. I don't think, however, it's necessary to go as far as Citizendium is currently planning on going, by requiring real names and biographies of all users. My thinking is that if an article is synthesized by 100 monkeys with typewriters but the finished product is giving the blessing of a credentialed professor of physics, it's pretty much just as reliable as if the professor had written it themselves. And if the same article gets the blessing of multiple credentialed experts, it could justifiably be considered more reliable than many printed sources written by a single author. The point is that the credentials that matter, are those of the people who stake their reputation on the accuracy of the article, not necessarily those of the people who contribute to it. So on this front, I think that while Wikipedia asks too little of users' backgrounds, Citizendium's current plan would ask too much, because as long as you have the credentials of one person who has signed off on an article, collecting non-verifiable bios of the article's other contributors doesn't actually gain anything.

The other side of verifying credentials is the use of credentials to prevent vandalism. In this situation it's not necessary to verify that the user actually is who they say they are; the system only needs to ensure that the same user is not signing up over and over again after previous accounts get banned for abuse. (You could ban users by IP address, but tools like Tor make it easy for users to connect from what appears to be a different IP address every time.) A blog post from Citizendium founder Larry Sanger lists three possible approaches instead: (a) requiring existing user X to vouch for new user Z before Z can join; (b) requiring new user Z to provide a link to a "credible" Web page establishing their identity; or (c) requiring new user Z to provide a link to a "credible" Web page of some person X who can vouch for Z's identity. I don't know how quickly a system could grow by referrals only -- after all, I was surprised that GMail took off so quickly during the period when you could only join with an "invite" from an existing user. Then again, GMail was giving away something for free that almost everyone could use, so most people who wanted it, would find themselves closely linked to someone else who had it. Citizendium, on the other hand, asks not what they can do for you but what you can do for them, and so might not achieve enough penetration to spread by referrals only.

I suggested that one alternative would be to send a postcard to each new user's physical address with a unique six-digit number, which they would have to enter in order to complete their registration, in order to verify that new users really were unique. The problem here, apart from the privacy concerns, is the delay that users would incur before their registration was complete, which would take away the "instant gratification" that they could get from starting to contribute right away. (You could let users edit before their address is verified, but that would just enable the same person to keep re-creating new accounts with unique but fake addresses, and use them to commit vandalism before the account was found out.)

Another idea would be that for new users, their first, say, three edits would go into a queue to be reviewed by verified users, and once the first three edits have been approved, the user is able to make edits in real time. (Since anybody would be able to review a new user's edits to make sure they were not spam, the new user's edits could be reviewed very quickly, since any Citizendium volunteer who was online, could review the latest entries in the edit queue and approve them.) It's true that a user could game this system by, for example, submitting three minor improvements, and then using their unblocked account to vandalize articles while they're being worked on. However, even in this case, the "vandal" would probably end up having a positive contribution to the site, because of the three small improvements that they'd already made. If a legitimate Citizendium volunteer would have to spend more effort making those three small improvements, than it would take to let a new user make those constructive changes and then ban them and revert their destructive changes once the user is caught committing vandalism (and the latter wouldn't take much effort at all), then Citizendium has actually gotten a good deal out of the "vandal"! (To make this work, a user's first contributions could not be "neutral" changes like replacing one word with a synonym; they would have to be actual improvements, even small ones, thus ensuring that the net effect of a potential "vandal" is positive.) There may be other possible solutions. These are just alternatives in case the model of referral by trusted users turns out not to work.

Now switching to the other side of the reliability issue: Whether the default article that is displayed to the public for a given topic, should be the latest "stable" version approved by credentialed users, or the very latest version incorporating all edits submitted by any user whatsoever. Having talked with members of the Citizendium and Wikipedia communities in their respective forums, there appear to be three schools of thought on the article stability issue. The first is that the whole idea of putting articles into an "approved" state and moderating all changes going forward, goes against the "spirit" of wikis in general and Wikipedia in particular. The second, suggested on the Wikipedia discussion list by Sheldon Rampton, is that it would be a useful feature if credentialed users could select certain page versions in the page history and "sign off" on the accuracy of one of those past versions; the page displayed by default would be the bleeding-edge latest one (with all of the possible vandalism and inaccuracies that entails), but users who wanted a reliable, citable source could look in the history. The third school of thought is that reliability is so valuable, that the default page displayed to the public and carrying the stamp of the project, should be the latest version approved by credentialed editors -- the model that Citizendium currently has in mind.

I'm not really partial to the first view, since I think the success of the project should be defined by how it achieves its goals (whatever you define those goals to be) and not in whether it kept with its original "spirit". Since Wikipedia has far more readers than contributors, if your motivations for contributing to or maintaining Wikipedia are at all oriented towards doing good for other people, presumably meeting the needs of readers is more important than keeping the party going for contributors (provided, of course, that the environment for contributors is at least pleasant enough to keep them contributing). The choice between the second and third points of view is more interesting. There's no obvious best-of-both-worlds choice here, because what motivates many contributors (the fact that their changes go live to the entire world, right away) is also what motivates vandals.

On the other hand, the problem doesn't sound unsolvable. You could go with the Citizendium model of editor-approved changes but create a prioritized system for "urgent" updates, in the case of changes to an article made to incorporate current events. Suppose users (who have been verified using one or more of the methods above) are each issued a certain number of "credits" that they can use to mark a proposed update as an urgent, breaking change. (Misusing these credits to mark changes as "urgent", that really aren't, would be considered abuse tantamount to spamming or vandalism.) Then let's say, for example, Anna Nicole Smith dies. A user could submit this change to the Anna Nicole Smith article, along with a link to a reliable news source (e.g. a wire service story) and a credit marking the change as "urgent". Since an editor would not need any particular expertise to view the article and verify that the change was accurate, any editor could review the "urgent request queue" and approve that particular change for publication, ensuring that the queue was checked frequently throughout the day and urgent updates would get pushed through quickly. Thus the site could keep pace with breaking current events without the kind of inaccuracies that plagued Kenneth Lay's Wikipedia entry when he died.

So there's a trade-off there, between displaying all the latest changes by default and motivating people to contribute but also running the risk of vandalism, versus displaying only the latest editor-approved page. Where there is not a trade-off, that I can see, is in the option of simply having an editor-approved version of a given page -- whether it's displayed by default, or only stored in the version history where people can look for it. To me, both of these steps seem to consist of pure gain for relatively little effort:

1. Verify credentials of academic professionals by poking their .edu address.
2. Allow them to give their "blessing" to certain versions of a page in the page history, so that users can rely on those specific page versions and even cite them as sources where appropriate.

So I hope that Citizendium will help bring more prominence to the idea, and that something similar might get incorporated back into Wikipedia. The approval of an identity-verified expert can improve an article's value so much, for such comparitively little extra effort, that it makes no sense not to have that option.

Comments

ok I'll bite

[Coraon]

here is a question, what creditationals do you need to report someones death? or how about reporting the plot line of a TV show? I mean do I need the nod of the TV geek from beat the geeks or something? I dont mean to poke fun at the issue here but lets be honest, if I wanna say that the number of bears is on the rise in the wild I can convince someone with the cred I need to do it for me... The power or Wiki is that anyone can edit, so anyone can fix the mistake.

Re:ok I'll bite

[Seumas]

First, let's acknowledge that this incident was a non-issue. Does anyone on this planet actually believe that it wasn't Sinbad or his publicist who edited the Wikipedia entry in the first place?

That aside, I'd like to point out that they are focusing on the wrong problems here.

Incorrect information that everyone knows is incorrect will swiftly be corrected. That isn't a problem. The real danger is when information is correct, but not recognized by the majority as being correct. You then end up with false information certified by the uninformed, mislead or simply incorrect majority.

The other large problem is the number of articles regarding people or subjects that are controversial and stir up emotions on both sides of the issue. The Wikipedia community becomes so bogged down in debating the subject and trying to ensure that the resulting article is completely devoid of anything even remotely biased-sounding to them that valid and reasonable information (like comments Anne Coulter made which caused a lot of controversy) are left out entirely. Some articles become incredibly bland and lacking so as to appease everyone.

It would seem that facts are facts and there should be no limit on the number or details of facts that are appropriate for a Wikipedia entry, as long as they are included in a sensible format. Instead, the attempt often seems to be to include the top ten percent of information about a subject and leaving out the rest. I presume the expectation is that the rest of the information will always exist elsewhere, so just provide a quick rundown on Wikipedia. Seems counterintuitive to the entire premise of a worldly collection of information.

I'm a huge wikipedia fan, but as I've stated - the general consensus agreeing to edit-out valid information is a greater fear of mine than the unlikely situation of invalid information somehow escaping the thousand-eyes.

Re:ok I'll bite

[eln]

You make a good point about subjects where no credentials exist (a PhD in Happy Days-ology?), but this is an imperfect solution even when credentials are available.

Even if credentials are valid, that doesn't stop people from making errors. If a credentialed professor puts his stamp of approval on a piece of information on the Wiki that he mistakenly believes is correct, that makes it a lot harder to correct that mistake, because the professor will be seen by most of the editors as the authority on the subject. Even credentialed professors are not infallible.

Also, what about a professor with a PhD in one field posting inaccurate information in another, closely related, field. The professor may honestly believe his information is correct, but that doesn't make it so. In this case, the incorrect information will stay on the Wiki until someone with more relevant credentials can successfully argue against it.

The only way to make sure information is correct (or at least in line with current academic thought on the subject) is to backup every statement of fact on the Wiki with a peer-reviewed authoritative source, such as a study published in a reputable journal or something like that. While this may sound like an ideal, I don't think it's really a reachable goal if you want to maintain the vast amount of knowledge that Wikipedia currently maintains.

The issue here is whether Wikipedia wishes to remain true to its original goal of being a community-edited encyclopedia or not. If so, it has to deal with the problems that come from that, and which are exacerbated by enormous popularity. The more popular you are, the more "undesirables" you attract: vandals and people who are trying to use your platform to spread an agenda. The only way to effectively combat this is with dedicated editors willing to spend an enormous amount of time policing the site for vandals, and doing research on disputed statements of fact. Trusting credentials, even verified ones, rather than research is a shortcut, and one that just isn't going to work as well as people seem to think it will.

Re:ok I'll bite

[nine-times]

Yeah, I don't really like the idea of credentials and such. The truth is that the whole topic is much stickier than lots of people want to believe. For example, being a PhD on a subject does not mean that you're unbiased regarding that subject. It doesn't mean that you're incapable of being wrong about that subject. It doesn't mean that there aren't non-PhDs who know more than you do.

"Facts" are often more complicated than they seem. One way or another, we can usually trace most of our knowledge back to some authority figure telling us, and we accept a lot based on authority. However, "authority" can easily be wrong, and often is. I actually rather like the idea of Wikipedia contributions being relatively anonymous. "Appeal to authority" is listed among "logical fallacies" for a reason. If your point is good, if you're correct and you have the background to argue your point well, then a know-nothing shouldn't be able to stand against you in a debate. If you can't debate your point, and you need to fall back on, "I'm a professor at [such-and-such] College!" then you probably don't really know what you're talking about anyway.

Identifying users has a good purpose-- to track who is making good contributions and who is making bad contributions. Citations are useful for determining where information is coming from. But does it really matter who is actually making the contribution? Do we really want the Wikipedia to be based on authority, rather than on demonstrably good information?

I like my privacy

[davidwr]

1) Not all experts have an EDU address
2) Many experts with institutional addresses can't or won't get their employer involved in authenticating them
3) "Underground experts" such as black-hat security experts value their anonymity greatly.
4) The same goes for political dissidents who have expertise to share under a pseudonym.

On a site like Wikipedia, some people will choose to post their biographies on their user pages and provide ways to contact them through "verifiable" email addresses such as an .edu address. Others will rely on the reputation they develop within Wikipedia or among several web sites where they use the same psuedonym.

Switchboard call...

[MS-06FZ]

"Hello, could I speak to Professor prof_guy_303 please?"
"Sure, please hold while I transfer you."

I Concure...

[Anonymous Coward]

As a world renowned proctologist, I welcome this type of identity verification.

Sincerely,
Dr. Seymour Butts

A couple of problems

[cyclop]

Really nice, and I mostly agree. Basically, you are asking for peer review for Wikipedia, that's something I really want.

I see three main problems:
- A .edu address is not a good technical solution. I am a Ph.D. student in Italy, and we don't have .edu addresses (my university address is @unibo.it). OTOH, I don't know if ALL .edu addresses come from respectable institutions (I remember I heard that some diploma mills had .edu addresses)
- There are subjects that are basically hard to be covered by academic institutions. Internet fads, TV series, web comics, urbant legends... What kind of academic peer review can be done on these articles? (Yes, they are important articles IMHO. They make of Wikipedia a resource that a traditional encyclopaedia cannot be).
- On the other hand, sometimes someone doesn't need to be a Ph.D. to be autoritative on a subject. A 16-y.o. hacker can be more autoritative on some software details than an informatics professor.

What are Bennett Hazelton's credentials here?

[writertype]

A bit of an irony here, since we're talking about authorities on a particular subject. My quick Google search turns up Hazelton as a twenty-something computer programmer who runs the Peacefire web site about filtering software and how to circumvent it.

That's all fine and good, but I'm not seeing how that qualifies him for an editorial on the Slashdot site. Or is he just a friend of an editor?

Not trying to troll here, honestly. I'm just curious why he was given the soap box to stand upon.

Re:What are Bennett Hazelton's credentials here?

[nine-times]

Good point. If he doesn't have a PhD in Wikiology, I'm not listening to him.

Wikipedia's Problems Are Hardly Unique...

[ausoleil]

Bias. Slander/Libel. Misrepresentation of academic credentials...and other malfeasances. These are unique to wikipedia? Hardly. These very things happen in the mainstream mass media from outlets we all know -- for example, the New York Times. Have they forgotten about Jayson Blair?

Blair is only one example of many. Fox News has had David Milloy, discredited author of junkscience.com on their payroll for years. Reuters has been shown to doctor photographs of Beiruit. And so forth and so on. Yet these organizations will tell you that they maintain the highest standards, and that they can be trusted. Thing is, their history shows that they make mistakes too. That they have been burned by liars and miscreants in their employ.

So what's the real issue here?

It has to come down to money, somehow, somewhere. Wikipedia is a free open-source reference center that sees widespread usage. This surely has to displease those that operate similar services in the for-pay space.

Yes, wikipedia needs to evolve and put in controls to limit vandalism, bias and academic fraud. But that does not imply for one second that other sources are any better and that they are free and clear of these problems themselves.

Credentials, schmedentials

[Dachannien]

Credentials shouldn't matter on Wikipedia. People aren't supposed to post original research there, so we shouldn't have to take anybody's word for it that they're more correct than someone else because of their credentials. Cite your sources so that other people can evaluate them, and do a good job of interpreting those sources for the layman when you edit an article, and you're doing your job.

Besides, how many Wikipedians are experts in a field, but never purport publicly to have a particular credential? Are those editors somehow less worthy of editing a technical article because they don't say they're a well-published physicist, even though they actually are? As long as Wikipedia doesn't require everyone to specify their expertise, credentials will be worthless.

putting way too much faith in what their job is

[Shivetya]

Sorry but to simply make an article out to be good based on a person's current job or education doesn't cut it for anything but hard sciences. When it comes to soft sciences, politics, religion, or hell even fashion, who is going to be the deciding factor?

Whats to prevent some anti-(insert favorite group) person getting in with credentials? They can then prevent edits to articles they get stewardship over. What if the group you use to determine an article's accuracy all has the same view? Its not hard when you get into politics. Just assuming they will be fair is meaningless when they can determine what is fair.

Sorry, I doubt it will be better than Wiki. Wiki is at least being held to new levels of scrutiny because they screwed up. This new one is just trying to gain traction by pointing towards the successful one and going "look at me, look at me" and spouting good sounding, at the surface, ideas in hopes people will pay attention.

I'll stick with Wikipedia for now. I know where their bias is and avoid those articles.

Re:This should surprise nobody

[nine-times]

You know, I really don't think it matters whether the owners are old, rich, or white, but I think you are onto an insightful issue. The real issue is that these media companies are built to be a method of disseminating information, and the people who are making careers in these companies view themselves in a particular way. They seem to want to think that they're the real source of information, and that they have a mission to make the world see things in a particular way.

Whenever you build a system and have people devoting their lives to that system, those people will view opposing systems (or even things outside their system) negatively. There are plenty of people within the big-media system who view the Internet as a threat to their mission. It's an alternative source of information which they don't get to filter, reframe, or control in any way. The most they can do is try to participate, which still puts them on equal footing with some random 14 year-old who just started a weblog. Since they've devoted their lives to the thing, the idea of being put on equal footing with everyday people is insulting.

Wikipedia is fun, but that's it.

[geek]

Every professor I've had has warned us vehemently not to use Wikipedia. It's useless for scholarly work as you have no idea if the material is plagerized or just down right incorrect. I've come across multiple errors myself, especially concerning some of the more subjective material. To use Wikipedia for scholarly work you would have to double check virtually every word, defeating the purpose in the first place.

I view Wikipedia as a fun tool and nothing more. You may or may not be getting the right info but regardless, it's still better than word of mouth. So long as people understand its place I don't have a problem with it, but when people start linking Wikipedia articles like a Christian would link the bible I have to call them out on it. It is NOT a scholarly source, even if a scholar submitted something to it. I in fact met someone in a class who thought it was funny to screw with Wikipedia articles, simply knowing human nature as I do, I wouldn't trust it as far as I could throw it.

Re:Wikipedia is fun, but that's it.

[Shotgun]

It's useless for scholarly work as you have no idea if the material is plagerized

Scholarly work? Why are you using any sort of encyclopedia for "scholarly work". Maybe we're running up against a language barrier, but "scholarly work" would only be done with original text, and maybe authoritative analysis of original text. The best any encyclopedia could ever do would be to point you to where to look for original text. Wikipedia is not unique in this respect.

And what has the possibility of plagiarism got to do with anything? If the text is posted there without the original authors permission, how does the text become any more true or false?

.edu

[Arancaytar]

> In its simplest form, couldn't a person's academic credentials be verified by sending a confirmation link to their .edu e-mail

Sure, if you can name a single educational institution with an edu address outside the US. Like the government sites here in Germany, our universities use country-codes. Banning Quatar's single IP address is one thing; alienating academics outside America is quite another.