Slashdot Mirror
Daily Kos Pollster Made Up Numbers
jamie found a story up on Daily Kos revealing that the polling firm they had contracted with for 18 months, Research 2000 or R2K, apparently made up or at least manually tweaked its polling results. The blog published a preliminary report by a team of statistics gurus (Mark Grebner, Michael Weissman, and Jonathan Weissman), and it is an exemplar of clarity and concision. The team reports, "We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls." Daily Kos will be filing a lawsuit against its former pollster. "For the past year and a half, Daily Kos has been featuring weekly poll results from the Research 2000 (R2K) organization. These polls were often praised for their 'transparency,' since they included detailed cross-tabs on sub-populations and a clear description of the random dialing technique. However, on June 6, 2010, FiveThirtyEight.com rated R2K as among the least accurate pollsters in predicting election results. Daily Kos then terminated the relationship. One of us (MG) wondered if odd patterns he had noticed in R2K's reports might be connected with R2K's mediocre track record, prompting our investigation of whether the reports could represent proper random polling. ... This posting is a careful initial report of our findings, not intended to be a full formal analysis but rather to alert people not to rely on R2K's results."
Perhaps you misread the summary? The Daily Kos is not at fault here.
of course this will turn into a "bash the left" and a "bash the right" thread. when ideology isn't the point. polling is the idiocy in question
and the guy who manipulated the numbers is clearly an amateur. the way you do proper poll manipulation is LOAD THE QUESTION. you poll people with a question with the proper turn of phrase to lead them towards the answer you want. then, when you present the answers to the poll, you also cage the results in such a way to lead the audience in the way you want them to interpret the results
polling is fucking joke. all results from the left, or the right, is complete bullshit, and a waste of your time
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
+1 on this.
If Stephen Glass worked for a conservative rag like the National Review, he wouldn't have been fired, he would've been promoted.
You mean the way that NYT promoted Jayson Blair several times even though his superiors were complaining about his inaccurate stories, until it became public knowledge that he just made things up?
The truth is that all men having power ought to be mistrusted. James Madison
You're also lazy, and ill informed. You could have spent a fraction of a second (0.15 seconds) with Google to find about 3,860,000 results for the search term "Rasmussen bias" to discover that, yes, in fact, there is some discussion of this point.
I know there is discussion. Even your quote says differences "can emerge from legitimate differences of opinion about how to model the electorate" and FiveThirtyEight has, in the past, noted Rasmussen's surprising accuracy in predicting election outcomes. Your link would not support the GPP's description of "horribly flawed" to Rasmussen -- merely "hmm that's interesting".
Liberal? Conservative? Compare perspectives at Left-Right
Nobody expects the Daily Kos to be accurate. It would be like trusting Ann Coulter, Rush Limbaugh, Sean Hannity, or anything ever aired on "Air America" before it went bankrupt.
On matters of fact, they're pretty scrupulous, especially when it comes to owning up to their own mistakes, like hiring R2K.
On matters of opinion and ideology, well, it's a political blog. What exactly is an "accurate" opinion?
Proud member of the Weirdo-American community.
Olbermann continued to close his show with the number of days since "Mission Accomplished" right up until a month or so ago, when he switched to the number of days since the deepwater horizon leak started. Much of the left does not pull punches against Obama for taking his time extracting us from these debacles.
The far left idealists can get quite heated against Obama. Me, all I have to do is imagine how McCain would have responded, then after I've wiped off all the cold sweat and stopped gritting my teeth, I have no regrets about 08.
Someone had to do it.
They explain the criteria by which they selected R2K, and it seemed fine. (RTFA)
That they caught R2K at this, and were willing to expose it, while other polls have also exhibited some of these patterns and continued to be used by their clients, says more good things about dkos than bad.
Someone had to do it.
They did look at it critically. Research 2000 was fired by Daily Kos before anyone noted any impropriety in the figures, simply because the numbers weren't matching up with reality. Shortly after this happened, Grebner, Weissman, and Weissman approached Markos with evidence of deliberate impropriety.
Does Daily Kos have a responsibility to not promote questionable information as truth? Of course, and they've apologized for the situation. But keep in mind that this information is only coming to light because someone with sufficient statistical background took the time to pore over the data. That sort of expertise is hard to come by, which is the reason why smaller media/news outlets contract out to firms like Research 2000 in the first place!
It's only relatively recently that there's been much interest in the science of polling. Before the emergence of aggregation sites like FiveThirtyEight or Pollster.com, it was extremely rare that you'd ever see this kind of statistical analysis of polling data. The traditional method of testing a pollster's reliability was simple trial and error over a period of several elections. Really, that's *still* the primary method. If anything, Research 2000 only got scrutinized in this case because of the issues with their accuracy that led to them being dropped in the first place.
For me, it's not really a partisan issue, despite the highly politicized nature of Daily Kos. It has more to do with the size of the media outlet. I would expect a major news organization with dozens or hundreds of employees, like Fox News or MSNBC, to be able to detect problems like these very quickly. A relatively small blog with maybe a dozen part-time employees like Daily Kos, or Red State, or whatever, I'm more willing to give a pass. At least at first: I'd expect Markos to learn his lesson from this and be more proactive in ensuring that it doesn't happen again.
Sean Daugherty "I have walked in Eternity -- and Eternity weeps."
To an educated mind, snarky ad-hominem attacks do more to discredit you than your opponent.
I prefer rogues to imbeciles because they sometimes take a rest.
Every Single Health Care Reform Idea is... ... a Republican idea.
You seem to forget that the National health reform model is modeled after the Massachusetts one, instigated by Mitt Romney, a Republican.
The Republican Party is only out for itself. If it's their idea,
Oh fuckit. I'm not writing this for the billionth time. Fuck the GOP, Fuck the Teabaggers, and Fuck you and your fucking short term memory.
--
BMO