Slashdot Mirror
Experts Rate Wikipedia Higher Than Non-Experts
Grooves writes "A new Wikipedia study suggests that when experts and non-experts look to assess Wikipedia for accuracy, the non-experts are harder on the free encyclopedia than the experts. The researcher had 55 graduate students and research assistants examine one Wikipedia article apiece for accuracy, some in fields they were familiar with and some not. Those in the expert group ranked their articles as generally credible, higher than those evaluated by the non-experts. One researcher said 'It may be the case that non-experts are more cynical about information outside of their field and the difference comes from a natural reaction to rate unfamiliar articles as being less credible.'" That's the problem people face when 'everyone who disagrees with you is a moron'.
Historian A: "The Nazis were horrible awful people who killed and murdered millions of people during World War II. They created nothing but pain and suffering while seeking out total fascist control of the entire world."
Historian B: "Nazism is not a precise, theoretically grounded ideology. It consists of a loose collection of ideas and positions: extreme nationalism, racism, eugenics, totalitarianism, homophobia, anti-Semitism, anti-communism, and limits to freedom of religion."
Now the reason I put those two up there is because your average person (I'm American so I may be biased on 'average') would probably favor historian A's perspective as opposed to historian B. Historian B is actually an excerpt from the Wikipedia entry. It's more encyclopedic as it's not opinion oriented. I'm not saying Wikipedia is free of opinions but what I'm proposing is that non-experts have an opinion and often when they read something that doesn't align with that opinion, they consider it to be incorrect.
The (on average high) neutrality of Wikipedia is most likely what causes non-experts to rate it as more erroneous than experts. Since the sample set was so low (as the report notes) then it is perhaps more likely that this happened.
I think that this is what the "Everyone who disagrees with you is a moron" article is getting at. I'm guessing experts are training not to suffer from that disease.
My work here is dung.
As everyone else but they know a little bit more about the process through which their own expertise derives. One need only read professional historians to understand that they have as much an agenda as anyone else for example.
Wikipedia is used all the time in the IP lawfirm where I work. If we need a definition or a quick rundown on a field before filing a patent, it's a good, well linked source.
science is a religion
that just by being a grad-student or a research assistant you become labeled an expert!
"Just Smile and Nod." --Huck
Could this just be a case of someone saying, "That can't be right!" only because they don't know if it really is?
Why is it when Microsoft/oil company/tobacco company is torched whenever they release a study saying Windows/gasoline/smoking is good because they are paid off blow-hards serving their masters but a Wikipedia study saying their articles are accurate (and make no mistake, that is what they are saying) doesn't raise an eyebrow?
These articles could have been written by anybody. It only seems appropriate that I would be skeptical about a topic written by a less than credible source about a t0pic I know little about.
The same is true for reference books, articles, television programs, etc. That's what the references are for. I agree you should be skeptical of wikipedia articles, I'm just not sure you should be more skeptical than you are of other sources of info.
I don't understand the people who attack Wikipedia....
It is free, a lot of people have put a lot of effort into it, and it is incomparable to any other repository of knowledge known to man.
Why the fuck would anyone want to piss on it? Don't like it? Shut up and go to a library.
I just hope that those non-experts didn't feel the urge to "fix" anything.
Because corporate-sponsored studies aren't editable by the public. People do raise an eyebrow with regards to Wikipedia, but any person with true knowledge can have a say in the content of an article. Plus there is clear public debate. No one can publicly debate or dispute a corporate study before it's published. Anyone can criticize it afterwards, but those disagreements never become an addendum to the study.
Developers: We can use your help.
... that when it comes to academic articles (e.g. physics) the only people who know enough math/jargon to get it close to right are the academics. So, the acuracy is of course going to be fairly high.
BUT, when it comes to policitically charged articles (or other non-academic articles), b/c of people's "MY true is reality no matter what the facts say" mentality nowadays, the acuracy plumits.
Basically, this study is nothing but a false positive in favor of wikipedia.
Isn't this the same criteria used for "well-respected, peer-reviewed journals"? You can abuse any such journal, just as wikipedia sometimes is.
However, wikipedia is different from such journals because it is a commons which is shared by people with differing viewpoints. It doesn't get the same bias that some journals may get where submitters and readers gravitate towards one of several different publications with slightly different biases (e.g. some journals favor publishing articles related to global warming as a concequence of human activities while others favor articles about it being a more natural phenomonon).
Debate is healthy, as long as it is reasoned. Wikipedia's nature enforces reason on debates about its contents. If a wikipedia entry gets edited by a person with a bias, a person with an opposing bias deals with it directly by editing the _same_ article, instead of proposing an alternate view somewhere else where it may not be seen by readers of the article. This beats the status quo , where oposing sides tend to just keep shouting their message without having any true debate.
science is a religion
Let me just say that I am so tired of the the rampant bias against wikipedia in education. I have had teachers go on 10 minute rants on how horrible of a site it is. I also am frustrated with the fact that during these rants generally there are no facts, studies or examples given to why they believe wp is untrustworthy only that anyone can change it so that means it is bad. Are there bad articles in wikipedia? Yes I dont think anyone would disagree with that. Are the bad articles the ones you will be looking at? I think thats the more important question. The more popular a topic is the edits it receives and the more trustworthy the information is. That is ofcourse ignoring the fact that now many big wikipedia articles cite sources. Another baseless concern is that at the time you are looking at the article some random false fact will have been inserted. Wp has this little feature called "history" I always check the last couple changes to a article before citing it in a paper. If something seems fishy I will cite a earlier version of the same article.
Anyways I guess in summary people are way too afraid of the wiki model.
Dooom
I tend to take most things I read on Wikipedia that I'm not an expert on with a grain of salt, simply because I keep finding errors in articles that I am.
The expert says "there are some good ideas behind this really shitty writing", and the non-expert says "wow, this is some really shitty writing." So the expert comes away with a higher opinion.
One of the things that one learns while doing his/her PhD is that he/she is NOT an expert in ANY field. It is only a matter of time for some big-headed know-it-all grad student to get crushed in a conference by a more experienced, better informed researcher. Being a grad-student and having research as your job makes you more open to new ideas and other people's opinions.
When you daily come accross many different approaches that try to solve the same problem, you are bound to learn that you must examine them all first before you decide. Otherwise you might miss a good idea that may eventually cost you your PhD. Sure you will have a favourite in the end, but that will be only after giving way to every possible option.
So a grad-student reading a Wikipedia article with an "alternative" (i.e. mistaken) point, would say "Hmm.. why not?", while a non-grad-student could say "WTF is this?" Of course, this would be the case only when the point is more close to being debatable and not obviously wrong.
I got into this discussion with some people on another forum the other day. There's a lot of people who regard it as little more than a repository of useless information, but it seems to me that that's more a factor of what sort of information they're looking for. There's a lot of things one there that I personally find pretty trivial, but who cares? It's not like having an exhaustive list of all the Pokemon characters is bothering anyone.
Personally, I find it to be a very useful resource for information on technical topics outside of my field of specialization. I do lots of modeling and conceptualization for games, so it's reeeeally nice to have an easy resource to explain the basics of say 19th century steel production or aircraft engines from the 30s. It's also really cool just to be able to read about a historical event and click a related topic to trace a thread through time. It's not a complete resource, but what is?
Um, I don't think being a Biology Ph.D. candidate makes one an expert at astronomy. If you try to pull that stunt in court as an expert witness, the judge won't like it.
They're not saying that, and that's not the "stunt" they "[tried] to pull". They're saying that the biology Ph.D candidate is an expert in biology, and he, as an expert in biology, rated biology articles rather high as far as accuracy goes. He then rated astronomy articles (a field in which he isn't an expert) lower. Now, move on to the guy who is a Ph.D candidate in astronomy, and you end up with opposite results (biology articles rated lower than astronomy articles). They weren't testing grad students against non-grad students, they were testing grad-students of different disciplines against each other.
my pet machine
If you're not an expert, you should be skeptical about your sources. In the case of Wikipedia, you should find an actual expert you can trust, have them read the entry, and tell you their expert opinion of its reliability.
Also, note that these experts aren't necessarily saying that Wikipedia is 100% accurate or reliable. The real issue might be that where a non-expert might mistakenly disregard a large amount accurate information from Wikipedia, an expert might understand that while the majority of the information was accurate, a few important inaccuracies were also present.
Any sufficiently well-organized community is indistinguishable from Government.
"Caution--and further research--needs to be used before citing anything learned from Wikipedia as a fact."
Yes, well, caution--and further research--needs to be used before citing anything learned from the Encyclopaedia Britannica... or the Handbook of Chemistry and Physics... or the World Almanac as a fact.
All of these are secondary sources. All of them are highly useful and are used as actionable sources of information every day, but none of them would be an acceptable citation in a research paper.
Furthermore, Wikipedia has always had policies that all information in Wikipedia must be derived from a published "reliable source" and that the source should be cited. Although these policies have mostly been honored in the breach, in the past year or so there has been an increasing tendency to cite sources explicitly. This is virtually a requirement for an article to become a home-page "featured article," for example. In some cases it is easier to trace the source of a fact in a Wikipedia article than in a traditional encyclopedia.
"How to Do Nothing," kids activities, back in print!
Actually the opposite of an expert pretty much covers most of Wikipedia.
Tubby or not tubby. Fat is the question
For the endocrinology classes I teach at the med school here, the most popular reference for both the students taking the class and the guest lecturers seems to be Wikipedia...I've even regularly seen physicians use the Wikipedia article as a refresher on a subject.
Reference books and articles (in my industry at least) are peer-reviewed, if you are getting them from the major outlets. You know they are credible, or at least validated by several other PhD's in the field.
Or maybe they are corporate funded propaganda. You don't know until you check the references and see who has peer reviewed them. The exact same thing goes for Wikipedia articles. Maybe articles in some given publication are always reviewed by certain parties and you can build up a level of trust, but said publications change and are sometimes purchased outright. You can't rely upon blind trust in a publication be it wikipedia or anything else. Look at the references and read critiques if you are a scientist. If, on the other hand, you just want someone to tell you what to think, you can ignore them. I just don't think wikipedia is all that less trustworthy that a random encyclopedia or book from the store and statistically that seems to be the case from the studies I've read.
1. You have a small inkling of the subject, and you are using the Wikipedia article to enhance your understanding.
2. You verify all statements in the Wikipedia article by reading all the primary source references. If the article has no references, discard it as a claptrap of lies.
#1 will enable you to spot the obvious (possibly deliberate) inaccuracies. #2 is to ensure the validity of the information. The article should be considered a secondary source, but its references (which every article should have) should be considered the primary source.
Increasingly people don't trust things that they aren't very familiar with because of the sort of political, under-handed, deceptive crap that has crept into so many areas of knowledge from the political world. Most people I know don't trust the mainstream media anymore and that ranges from people who are nearly communist in their left leanings to people who are practically John Birchers. Dispassioned, reasoned discussions are rare these days.
Think it's not the problem with even science? Why do so many people attack Bjorn Lomborg with a fanatical ferocity for daring to raise scientific questions about how, why and if global warming is happening? Why can't people who claim to operate on civilized values like reason sit down and have a friendly chat. "Interesting, Bjorn, let's look at your facts; Hmmm, interesting, but I don't think you considered the following (X, Y, Z); Touche, but I would like to present this, this and that to prove that global warming is not human-caused." Instead it's more like, "YOU MOTHERFUCKING ASSHOLE WHO ARE YOU TO QUESTION ANY ASPECT OF GLOBAL WARMING?!"
The truth is that there are so many people who are significantly maleducated today that it's no wonder why people are screwed up. I mean, it was a real eye opener for me, when I started reading up on my own time, about some of the cultural practices of the ancient world. Most of the people who look horrified at religion today have never even heard of such practices as Pater Familias nor know that their celtic ancestors (if that applies to them) often practiced human sacrifice. I honestly think that based on some of the conversations I have had since I started doing these things on my own, that the maleducation of the American public today is worse than the lack of education that existed 200 years ago. There is nothing worse than having a horrendously bad education--it'd be better to simply be a void that can be filled by actual knowledge.
Now, the reason that I brought up the global warming issue was not to beat a popular pinata, but to illustrate the fact that to many "laymen," the "experts" often come off as narrow-minded fanatics. That doesn't inspire confidence in the average person. What does inspire confidence is a calm ability to articulate on his or her level with facts that back it up. Problem is, too many people have an agenda and too many people are too caught up in it to be convincing to the majority who won't immediately accept what they say at face value as though it were penned by the hand of God.
My experience with tradition media is that almost always when there is an article on something that I actually know a great deal about, they get many facts/details wrong and those stick out to me. Even if the overall story is basically OK, it is always troubling when there are significant numbers of obviously wrong things. In general, of course, this erodes my confidence in coverage of things outside my areas of expertise because there is no reason to think that reporters make mistakes only when they are writing about something I know well. So this study could be coming from noticing the same effect- even if the gist of the article is OK, experts notice problems and then become suspect overall.
There are regular stories on Wikipedia on Slashdot, and occasional stories on other wikis. Shouldn't there be either a Wikipedia icon or a Wiki icon to distinguish these stories? The Wikipedia "multilingual globe being built" is copyright (one of the very few things in Wikipedia which is) so you can't use that, but the Wikipedia "W" is fairly well known. Looking through Wikimedia Commons, this puzzle piece looked good to me. I don't know if the GFDL licence would be a problem for Slashdot.
The MediaWiki sunflower would only be suitable as an icon for Wikis powered by that piece of software. I don't have an idea for an icon to represent all wikis.
It appears that Goodwin's law is not invoked for the first comment in a discussion. This come logically from the requirement that Goodwin's law apply to a discussion that involves the Nazi/Hitler example as a means to refute another comment.
Being the first comment, an off-by-one exception occurs, resulting in an aborted termination of the thread.
science is a religion
There's no guarantee that an article, at any moment in time, even approaches accuracy. If an expert in a field has reviewed some piece of information within it, perhaps a mechanism allowing him to digitally sign that piece of information would allow the article to gain some credibility.
In theory, citations should achieve the same goal, but it's clear that people don't want to research Wikipedia articles that have already been written. They want to use them as research. Do we want to work to try to change people's habits and perceptions, or change the system to work with people's habits and perceptions?
Experts understand the subtle nuances of a subject and therefore qualify their position with lots of 'if' or 'buts'. An informed observer appreciates these nuances. An uninformed observer does not, it appears less precise and less clear.
The less competent see fewer nuances and therefore make more straight forward assertions, they qualify their position less, therefore it looks clearer to an uninformed observer.
While I think your test protocol would work better, I still have to take issue with claiming 55 articles is not statistically significant sample size. Even tests with 1 data point are statistically valid, just with no confidence. 3 is a nice start if you have variable data. You can start making some statistical predictions. 30 is a common sample size used by people, but only because s and sigma are closely converged. My point is that statistics don't care about the sample size. Significance is in the eye of the beholder and is determined independent of sample size (and hopefully set a priori). Your chances of the results being significant increase with sample size, but the sample size itself is neutral.
The world is made by those who show up for the job.
Oh. Sorry. Left my tinfoil hat at home.
Who needs a tinfoil hat? Have you been living in a cave? It has been standard procedure for many large companies to fund the publication of "scientific" studies for many, many years. If you aren't aware of it, the chances are you have read some unknowingly. Heck, it is not even uncommon for "news" programs to run advertisements made to look like news announcements in the middle of the news with no disclaimers.
t least I know when I submit my papers to the AIAA they are peer-reviewed by PhD's and that when I pick up a journal from them, those papers have been too.
No you don't. You trust they have been, but you don't know they have been. For all you know News Corp bought AIAA last month. And even if they have been peer reviewed, maybe those PhDs were incompetent twits. I've met enough of them in my day to know they are plentiful. Or maybe PhDs who are very competent read the paper and wrote a rebuttal about how the data conflicts with both their data and that of several other ongoing research projects, but that rebuttal has not been published anywhere you read.
If you rely upon one source, without checking the references and without seeing what other journals have to say, you're just taking it on faith.
There have been cases of 'vandalism' per see where an expert in her field has posted a perfectly valid page on Wikipedia and then had it changed by someone who doesn't understand the underlying science.
Sure there have. And there have been cases of print encyclopedia having intentional lies and data that is simply wrong or out of date. The question is, does the many eyes many hands approach to wikipedia increase or decrease the chances of the information being correct? Statistically, it seems to increase the chances of the data being right, especially for certain topics.
A system that allows that is not a system that can be relied on for any meaningful data.
My point is, no one system can be relied upon for meaningful data. If you're going to rely upon only one system, however, you've not shown any evidence that wikipedia is any worse than average, aside from your unscientific conjecture that you think that is the case. Gather data first, then propose a conclusion, remember?
Think about medicine for a second. Would you rather have 500 enthusiastic amatuers doing surgery, each correcting something that the other thought was wrong, or would you rather go to a brain surgeon, who has studied medicine for years? The doctor has a much better chance of getting it right than amatuers.
Secondly, with experts, I can usually manage to figure out who they are and where they came from. Even using Wpedia's History, most users are 1337ster_275, or worse yet, an IP address. Well, that's not reassuring at all. Knowing that the author or contibuter was Thomas Gadfly, pH.D. from the University of Montana would mean something. Not the least of which is that he actually makes a living working in that field. He's studied the subject, not skimmed a "Physics of Star Trek" book and declared himself an expert. Who is 1337ster_275? Did he finish high school?
And with all due respect, how do you know enough to know if the Wpedia article is "fishy"? All that the History says is "it changed", so was it corrected, or was it reverted back to a wrong but popular version? It will always come down to trust, and since there's no way to check back on the various writers, it comes down to blind faith in the power of the consensus.
I could see Wpedia as a way to look for search terms on the way to getting the facts and legitamate sources. I could see using Wpedia in an internet debate where nothing was really at stake. Just not in research papers and the like where it's imperative that your facts are accurate.