Statisticians Investigate Political Bias On Wikipedia

Hugh Pickens writes "The Global Economic Intersection reports on a project to statistically measure political bias on Wikipedia. The team first identified 1,000 political phrases based on the number of times these phrases appeared in the text of the 2005 Congressional Record and applied statistical methods to identify the phrases that separated Democratic representatives from Republican representatives, under the model that each group speaks to its respective constituents with a distinct set of coded language. Then the team identified 111,000 Wikipedia articles that include 'republican' or 'democrat' as keywords, and analyzed them to determine whether a given Wikipedia article used phrases favored more by Republican members or by Democratic members of Congress. The results may surprise you. 'The average old political article in Wikipedia leans Democratic' but gradually, Wikipedia's articles have lost the disproportionate use of Democratic phrases and moved to nearly equivalent use of words from both parties (PDF), akin to an NPOV [neutral point of view] on average. Interestingly, some articles have the expected political slant (civil rights tends Democrat; trade tends Republican), but at the same time many seemingly controversial topics, such as foreign policy, war and peace, and abortion have no net slant. 'Most articles arrive with a slant, and most articles change only mildly from their initial slant. The overall slant changes due to the entry of articles with opposite slants, leading toward neutrality for many topics, not necessarily within specific articles.'"

Comments

Hope they don't do just word frequency analysis

[Anonymous Coward]

Or shall we remind them that the English Wikipedia is not only about U.S., and the word 'republican' and 'democrat' have other meanings too?

Re:Hope they don't do just word frequency analysis

[Anonymous Coward]

Would it kill you to read the paper?

We obtain a list of 111,216 articles. We then eliminate these articles that cover countries other than the United States.
[...]

For each of these articles, we construct a slant index by applying the methods and estimates developed by Gentzkow and Shapiro (2010), hereafter G&S. G&S select 1,000 phrases based on the number of times these phrases appear in the text of the 2005 Congressional Record, applying statistical methods to identify phrases that separate Democratic representatives from Republican representatives, under the model that each group speaks to its respective constituents with a distinct set of coded language. In brief, we ask whether a given Wikipedia article uses phrases favored more by Republican members or by Democratic members of Congress.

And the corresponding footnote:

The words “republican” and “democrat” do not appear exclusively in entries about United States politics. If a country name shows up in the title or category names, we then check whether the phrase “United States” or “America” shows up in the title or category names. If yes, we keep this article. Otherwise, we search the text for “United States” or “America.” If these phrases do not show up more than 3 times in the text, this article is dropped. This process keeps articles such as “Iraq War” but drop articles related to political parties in foreign countries.

Researchers do think of this stuff, you know.

Re:Hope they don't do just word frequency analysis

[EdgePenguin]

Indeed; I should imagine that those who fought against Franco in the Spanish Civil War have little in common with Newt Gingrich and Mitt Romney, and neither have much in common with the former Iraqi Republican guard.

Re:Hope they don't do just word frequency analysis

[TheRaven64]

'America' appears 7 times in the article 'Irish republicanism' (3 times as 'America' 4 in 'American') and so by their metric (must occur 3 or more times) it would go in, in spite of being nothing at all to do with the US political party of the same name.

Equally biased != NPOV

[TorrentFox]

One guy may say that the sun is green, the other guy may say it's purple. Having both of them in the same article does not make it neutral.

Re:Equally biased != NPOV

[Theophany]

You make the erroneous assumption that the biases are empirically provable or that opinions are in some way absolute rather than normative, which is not always the case. (Actually, in politics this is never the case, they all distort facts beyond any limitations of meaningfulness to suit their own agendas.)

Re:Equally biased != NPOV

[CastrTroy]

This is the same problem we have in the news. Reporters (some anyway) want to be seen as non-biased, so they give equal time to both supporters and non-supporters of global warming, and therefore the general public thinks that there is actually some kind of debate in the scientific community over whether or not global warming is really happening. Same goes for evolution and a lot of other topics. Sometimes it even gets a little out of hand, like this Anderson Cooper interview where he has some non-educated person who just embarrasses herself on national television, because they insist on having someone from the other side of the issue talk about it, and she was the only person stupid enough to try to defend the point of view. Ignoring the other side of the debate is fine if the other side of the debate is provably wrong.

Libertarian bias?

[EdgePenguin]

Surely they need to investigate libertarian bias (especially seeing as Wales himself is, how should I put this, a raving Objectivist nutjob). The fact that libertarian beliefs overlap with democrat and republican beliefs can explain the two separate slants with one single hypothesis.

Dangerous to Conclude this is "Slant"

[eldavojohn]

This is a neat study but I feel like the foundational assumptions are subjective and a little flawed.

(civil rights tends Democrat; trade tends Republican)

Could that be simply because Democrats invent/introduce/overuse new phrases and talking points for civil rights and Republicans invent/introduce/overuse new phrases and talking points for trade? For example, you'd probably hear Democrats say "Equal Opportunity Employment" or "Affirmative Action" a lot and you'd probably hear Republicans say "Laissez-faire" or "Free Market" a lot. What would be the antithesis of these phrases for the other side? I would posit that it's entirely possible that these articles are not on average biased and instead are merely explaining and using the phrases that each party has employed to tackle their number one priorities.

On top of that, I didn't see anything that seemed to indicate that they used windowing to determine when a phrase was opposed to the phrase they were using. For example if you found that the acronym ACORN indicates a Democratic slant but there's a whole section on its Wikipedia page full of negative criticism despite them using 'ACORN' frequently in that section. Would this section be identified as a Democratic slant?

Where is this G&S word bank? Where is the list of results so I can look up the ACORN article's scores?

I'd be more interested in the media

[Grayhand]

I would have assumed a fairly even distribution with Wikipedia so the results weren't that surprising. I'd be more interested in using it to find bias in the media. There are obvious cases of bias such as Fox but I've noticed a gradual move towards the right in groups like CNN who seemed in the past Democratic in it's leaning. I've found more open reporting from comedians these days. Some subjects only the comedians take on that the media avoids or barely mentions. One interesting trend I noticed early on is all media sources including supposed left wing groups call the President Mr Obama while Bush was generally called President Bush and I can't remember him being referred to as Mr Bush. Traditionally the media always calls a sitting President by the title President and expresidents are generally referred to as Former President. Pay attention when you listen to the news and see if I'm right. Both Bushs and Clinton are referred to as former Presidents far more often than Obama is called President Obama.

Re:How to write without political bias?

[vlm]

What can be done to avoid political bias and how do we do it consistently?

Don't write about political topics that are relevant to you?

For example writing about modern civil rights (gay marriage, gun rights, etc) in the USA is going to get a intention and/or unintentional bias from me.

However if I research and report on the political situation in France, where I have no dog in the fight, I'll probably end up pretty much unbiased.

Its a big interconnected world... there's really no reason for locals to have to write biased filler about local issues.

Doesn't have to be geographic. I have no personal interest in the gay marriage thing, not being gay or close to those in their subculture and not being hyper-christian, so I can be extremely unbiased about the topic. This SHOULD work, but it fails anyway, because my completely unbiased view unsurprisingly seems to match the (few) non-cowardly (D) and oppose almost all the neo-(R) so I'll be accused of being "politically biased" based on results, although I obviously don't have any reason to care that would influence the process of writing about it. Think of how everyone naturally decides that slavery was a dumb idea now, but it was a hot political football around 1860 or so in the USA.

There's some other hints, like if you find evidence of sloganeering in your writing you're probably doing it wrong.

Interesting, but shallow

[Hentes]

The frequency of using individual words is far from an actual political bias.

Re:Please limit it to articles about the US gov't

[Chrisq]

Europe has a clear political bias too (we're left wing socialist surrender monkeys), and we're quite happy about that. So, please America, leave our wikipedia alone..

Didn't you know that every use of British spellings counted as a "left wing" article.

Three accountants go duck hunting.

[drainbramage]

The first accountant shoots and misses a meter high.
The second accountant shoots and misses a meter low.
The third accountant says "Got it"!
------------------
It looks like another paid for study that proves what they were asked to prove.
They only had to determine which data points would produce the required end point.

Re:How to write without political bias?

[Hatta]

You can't. Even if you write purely factual prose, it's slanted toward's the left. Reality has a well known liberal bias.

And this article is example 1: Apodixis

[Anonymous Coward]

Bias is rhetoric. Apodixis: A rhetorical device that stealthily inserts a false pretense of general knowledge. For example "As everyone knows..."

Or, as this article does: "expected political slant - civil rights tends Democrat"

Republicans broke the Democrats filibuster of the Civil Rights Laws of the 60's. The Republican Party was formed for the sole purpose of overturning Democratic Legislation that allowed slavery to expand into the Western Territories. The first Republican President freed the slaves. Every Governor of every state that let loose the fire hoses on and dogs on minority students was a Democrat.

Study rhetoric, and don't fall for it. We are most vulnerable to the rhetoric we cheer for. That's where we should put most of our scrutiny.
Being tricked by adversary is bad enough, being tricked by someone you support is truly insulting.

Bias is rhetoric. Apodixis For Example

[TerryCary]

Apodixis: A rhetorical device that stealthily inserts a false pretense of general knowledge. For example "As everyone knows..." Or, as this article does: "expected political slant - civil rights tends Democrat" The Republican Party was formed for the sole purpose of overturning Democratic Legislation that allowed slavery to expand into the Western Territories. The first Republican President freed the slaves. Every Governor of every state that let loose the police, the fire hoses and the dogs on minority students was a Democrat. Republicans broke the Democrat's filibuster of the Civil Rights Laws of the 60's Study rhetoric; don't fall for it. We are most vulnerable to the rhetoric we agree with. So, that's where we should put most of our scrutiny. Being tricked by an adversary is bad enough, being tricked by someone you support is truly insulting.

Re:How to write without political bias?

[TFAFalcon]

There is also another problem. They are measuring only the bias toward the two main parties. What about bias toward/against other points of view?

Debates can be reframed, introducing bias

[Urban Garlic]

Bias can sneak in because of changes in terminology, presumably in both directions, although I've noticed it more on the right these days. As Robert Anton Wilson famously observed, you can go from liberal to conservative without changing a single idea if you wait long enough -- the reverse is also true, depending on the domain in which you have your ideas.

For instance, an article about taxation written in the 1990s might be considered neutral in its time, and talk about the "inheritance tax" a lot. Fast forward ten years, during which the term "death tax" has come into prominence, and the old term "inheritance tax" is only used by fogies and liberals. The textual analysis of the unchanged article will now score it as "liberal", because the terms of the debate have shifted.

This can happen with policies, too -- I remember when a carbon tax was considered a compromise position between liberals, who wanted to directly regulate carbon dioxide emissions, and conservatives, who felt that some kind of market mechanism would provide useful flexibility. Carbon taxes were a technocratic, ideologically neutral solution when they were proposed, but now they're seen as liberal social engineering.

It doesn't always go rightward, of course, some debates have been successfully re-framed by the left, as well, I think -- "global warming" used to be a neutral descriptive term, but the warming isn't uniform, so "climate change" is the preferred term, and I think it's mostly conservatives who use the term "global warming".

That ought to blow up my karma for a solid year...

Re:How to write without political bias?

[JWW]

Ok, I'll not mod the troll for what it is. But that condescending "reality liberal bias" shit is getting really old.

Just to point to a counter example. Greece would prefer to stick with its liberal policies and continue spending government money it doesn't have. In this case, and I am not claiming that this is true in all cases, reality has a decidedly conservative bias. Greece needs to make heavily conservative moves with respect to their government spending or they are doomed. And no, the liberal "raise taxes" move isn't going to work, either.

So, Greece is living in a reality that has a conservative bias. Through proof by contradiction, the lie that "reality has a liberal" bias (in all cases) is patently false.

Re:How to write without political bias?

[nedlohs]

There are no other points of view.

Re:How to write without political bias?

[1s44c]

There are no other points of view.

How come that didn't get modded insightful?

The two party bi-polarism has buried every other point of view in the US and it has pretty much killed the democratic process for years now. Nobody gets a fair vote in anything unless it can be represented as one extreme or the other. The system is rigged in such a way that this is unlikely to change and the media keeps dumbing down everything to the same two extremes.

A democratic system can't function with only the illusion of choice, you need more than just 2 viewpoints.

Re:How to write without political bias?

[tbannist]

That's an interesting point of view, but it's not a very good proof. Greece has been following the "conservative" prescription to solve their economic problems. The result? Record levels of unemployment. If you think the American economy is doing poorly, consider that Greece now has hit a new record level of unemployment (22%) and over 50% unemployment for people aged 15-24. Given how well the "conservative" solution is working out, do you think it's any wonder they're looking for a "liberal" solution instead?

More importantly, you missed the entire point of the joke. The joke is that when reality disagrees with conservatives, they claim reality is biased. It's a sad commentary on the far right's rejection of facts and embracing of fictions to justify their views in the face of evidence to the contrary*. Liberals, while not immune, are currently much less prone to that type of behaviour as recently confirmed.

* Sometimes it manifests as an inability to accept some of the relevant facts. For instance, Grover Norquist is unable to accept that Ronald Reagan raised taxes, increased the size of the government and tripled the U.S. federal debt during his term in office. This colours his entire perception of government and it's role in the economy, since he credits the Reagan economic boom solely to the tax cuts that Reagan implemented without considering any of the other relevant factors.

Re:How to write without political bias?

[tnk1]

Bias is an interesting phenomenon. I can be a fundamentalist Christian and think all gays are going to Hell to burn in eternal torment and still be okay with gay marriage as a civil situation since its not like the government can actually force me to accept that they are truly married in a Christian sense.

Alternately, I could also be atheist or agnostic, believe that being gay is perfectly acceptable and normal but still wonder just what the point of gay marriage is actually supposed to be, from a state perspective.