Daily Kos Pollster Made Up Numbers

jamie found a story up on Daily Kos revealing that the polling firm they had contracted with for 18 months, Research 2000 or R2K, apparently made up or at least manually tweaked its polling results. The blog published a preliminary report by a team of statistics gurus (Mark Grebner, Michael Weissman, and Jonathan Weissman), and it is an exemplar of clarity and concision. The team reports, "We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls." Daily Kos will be filing a lawsuit against its former pollster. "For the past year and a half, Daily Kos has been featuring weekly poll results from the Research 2000 (R2K) organization. These polls were often praised for their 'transparency,' since they included detailed cross-tabs on sub-populations and a clear description of the random dialing technique. However, on June 6, 2010, FiveThirtyEight.com rated R2K as among the least accurate pollsters in predicting election results. Daily Kos then terminated the relationship. One of us (MG) wondered if odd patterns he had noticed in R2K's reports might be connected with R2K's mediocre track record, prompting our investigation of whether the reports could represent proper random polling. ... This posting is a careful initial report of our findings, not intended to be a full formal analysis but rather to alert people not to rely on R2K's results."

Comments

To be fair...

[Anonymous Coward]

Nobody expects the Daily Kos to be accurate.

It would be like trusting Ann Coulter, Rush Limbaugh, Sean Hannity, or anything ever aired on "Air America" before it went bankrupt.

Re:To be fair...

[RyuuzakiTetsuya]

Actually, I do. And now They're suing the pants off of R2K.

If this was the National Review Online, or Free Republic, or what have you, there would be a huge push to cover this up and blame the "liberal media"(whatever the hell THAT is) for any accusations that they did something wrong.

Re:To be fair...

[spun]

Wrong. Now that teabaggers know what the term means, they call themselves tea partiers. But back in the day, they carried teabags around and called themselves teabaggers.

Here's an article backing up that fact, but I warn you, it is from that den of liberal iniquity, Billy Buckley's The National Review, so take it with the grain of salt that any reading of The National Review requires.

http://nrd.nationalreview.com/article/?q=Mjk1YmRjNzIxNmUwMTI0ZWYxZWU4OWU2MzFiOWJmNDE=

Re:To be fair...

[NotBornYesterday]

The snarky ad hominem bit was "teabagger". And don't pretend you didn't mean it that way.

Re:To be fair...

[Curunir_wolf]

Nope, that proves nothing other than some people are as ignorant as yourself. That article was written in December, 2009. And the author apparently didn't know anything about the Tea Parties that had been happening for almost three years - he seems under the (mistaken, or intentionally misleading) assumption it had something to do with Obama's election.

Here's some insight from some of the progenitor tea parties.

Re:To be fair...

[NotBornYesterday]

Like I said, it has no place in polite conversation, regardless of who uses it. Plus, many who are being tarred with this brush had no part in its origins.

And none of that changes my original point, which was that it is used as a pejorative term to attack people rather than engage in debate, and therefore is usually used by people whose ability to engage in an interesting discussion is less developed than their desire to mock those who think differently.

Re:To be fair...

[DJRumpy]

Actually no. They carried the very signs that started all of this:

http://washingtonindependent.com/69660/correcting-jay-nordlinger

In January of 09, they had a Facebook page that had some back and forth discussion about the 'alternate' meaning of teabag with some surprised disdain when they were informed as to what the term meant. They were apparently unaware at that point.

This is from the rally in DC on April 15th of 2009:
http://washingtonindependent.com/31868/scenes-from-the-new-american-tea-party

One final little tidbit...the debate by conservatives as to whether or not to wear the title with pride ;)

http://tpmdc.talkingpointsmemo.com/2009/12/to-teabag-or-not-thats-still-the-question-for-conservatives.php

Slightly misleading headline?

[Pojut]

Those who aren't used to phrases used with "political" centric organizations might mistake the title as saying someone who is on Daily Kos' payroll flubbed the numbers, rather than a company working on contract with them.

R2K not alone in this.

[Zephyr14z]

I used to work a different major polling company, and I can assure you R2K is not alone in just making up numbers. Easily 80% of surveys that went through my region were completely falsified, and the remaining 20% rarely matched the demographic they were supposed to be answering for. Survey administrators have quotas, and then get paid extra for additional surveys past that, but there is basically nothing done to verify any of the surveys turned in, and everybody in the company knows it. Don't always trust what you read, especially not statistics.

misleading headline...

[emagery]

For headline skimmers, this post would produced a completely inaccurate sense of what the article was all about... at length, the D.KOS are the ones who found out about this and are doing something something about it. That's good... but if you just read the headline, you'd come away thinking that D.KOS were the culprits instead.

Re:Give them credit.

[dkleinsc]

Apparently the situation with Rasmussen is complicated, but this seems to be a fairly decent starting place (that's not just some activist blogger).

You Are Not a Republican

[Gary W. Longsine]

You're also lazy, and ill informed. You could have spent a fraction of a second (0.15 seconds) with Google to find about 3,860,000 results for the search term "Rasmussen bias" to discover that, yes, in fact, there is some discussion of this point.

Nate Silver on Possible Biasin Rasmussen Reports
"What Rasmussen has had is a "house effect". So far in the 2010 cycle, their polling has consistently and predictably shown better results for Republican candidates than other polling firms have. But such house effects can emerge from legitimate differences of opinion about how to model the electorate. And ultimately, these differences of opinion will be tested -- based on what happens next November. If Rasmussen's opinion turns out to be wildly inaccurate, that will impeach their credibility, and believe me, we will point that out. Likewise, if they turn out to be right when most other pollsters are wrong, we will point that out too."

Re:Give them credit.

[Attila Dimedici]

The quote you gave merely says that the results that Rasmussen is getting at this time show a public more greatly favoring Republican positions not that Rasmussen is fudging the numbers to get those results. The rest of the article can be dismissed since the source for the article clearly favors Democratic policy over Republican

Re:Mark Twain said it best

[jkauzlar]

The headline on this article was stupidly misleading. Months ago, if not over a year, Nate Silver at fivethirtyeight.com called out R2K for just this one thing. You may remember Silver's interesting observation that the least significant digits in the polling results did not follow a normal random distribution. For example there may have been too many .9's in the results (58.9 or 63.9, etc) while there were few instances of other digits.

The pollster was subscribed to by DailyKos, among hundreds of other news organizations, and the results were skewed IN FAVOR OF RIGHT-WING CAUSES, not left-wing, so the assumption that DailyKos was somehow complicit in this is absolutely not true. (And I've rarely, if ever, read DailyKos, so I have no personal interest in defending them.. the headline is just grossly misleading).

Re:Mark Twain said it best

[skids]

That was Strategic Vision, not R2K.

(Hey, I'd be much happier if people named products with distinguishable proper names rather than generic sounding word combinations and worse yet, acronyms, so you have my sympathies for getting them mixed up.)

Re:I am not sure who those "teabaggers" are...

[Curunir_wolf]

Did you know the original Teabaggers were protesting the fact that the wealthy British lowered taxes on their own tea below the taxes on the colonial tea?

That's a pretty inaccurate depiction of the Tea Act and why the colonists opposed it. In essence, the British government was protecting it's own favored company (East India Company), in favor of other traders (and smugglers, because tea carried a hefty tax). So actually the colonists favored free trade instead of crony capitalism (or fascism, if you prefer), and when the British government tried to pass laws that provided monopolies for East India, the colonists rebelled.

I think that's a pretty good analogy with motivations of the modern-day Tea Party protesters.