Slashdot Mirror
Daily Kos Pollster Made Up Numbers
jamie found a story up on Daily Kos revealing that the polling firm they had contracted with for 18 months, Research 2000 or R2K, apparently made up or at least manually tweaked its polling results. The blog published a preliminary report by a team of statistics gurus (Mark Grebner, Michael Weissman, and Jonathan Weissman), and it is an exemplar of clarity and concision. The team reports, "We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls." Daily Kos will be filing a lawsuit against its former pollster. "For the past year and a half, Daily Kos has been featuring weekly poll results from the Research 2000 (R2K) organization. These polls were often praised for their 'transparency,' since they included detailed cross-tabs on sub-populations and a clear description of the random dialing technique. However, on June 6, 2010, FiveThirtyEight.com rated R2K as among the least accurate pollsters in predicting election results. Daily Kos then terminated the relationship. One of us (MG) wondered if odd patterns he had noticed in R2K's reports might be connected with R2K's mediocre track record, prompting our investigation of whether the reports could represent proper random polling. ... This posting is a careful initial report of our findings, not intended to be a full formal analysis but rather to alert people not to rely on R2K's results."
Nobody expects the Daily Kos to be accurate.
It would be like trusting Ann Coulter, Rush Limbaugh, Sean Hannity, or anything ever aired on "Air America" before it went bankrupt.
Those who aren't used to phrases used with "political" centric organizations might mistake the title as saying someone who is on Daily Kos' payroll flubbed the numbers, rather than a company working on contract with them.
Living With a Nerd
In this case it was Lies, Damned Lies, and "stuff we made up".
Whether or not there is some sort of god, I'm not supposed to say/god is a word and the argument ends there-Smog
Unlike the many Republican outfits which used partly- or wholly-fabricated polls by Strategic Vision, or the many media outlets which continue to use the horribly flawed Rasmussen polls to create eye-catching headlines, Kos immediately dumped the pollster, did an investigation, owned up to the errors publicly, and is now pursuing legal recourse.
This is exactly how you would expect an honest media organization (if one with a considerable political agenda) to behave. Too bad the mainstream media and those on the other side of the aisle don't seem to want to do the same.
You'd think they would spend Soros money more carefully.
Professional Politicians are not the solution, they ARE the problem.
I used to work a different major polling company, and I can assure you R2K is not alone in just making up numbers. Easily 80% of surveys that went through my region were completely falsified, and the remaining 20% rarely matched the demographic they were supposed to be answering for. Survey administrators have quotas, and then get paid extra for additional surveys past that, but there is basically nothing done to verify any of the surveys turned in, and everybody in the company knows it. Don't always trust what you read, especially not statistics.
For headline skimmers, this post would produced a completely inaccurate sense of what the article was all about... at length, the D.KOS are the ones who found out about this and are doing something something about it. That's good... but if you just read the headline, you'd come away thinking that D.KOS were the culprits instead.
Comment removed based on user account deletion
of course this will turn into a "bash the left" and a "bash the right" thread. when ideology isn't the point. polling is the idiocy in question
and the guy who manipulated the numbers is clearly an amateur. the way you do proper poll manipulation is LOAD THE QUESTION. you poll people with a question with the proper turn of phrase to lead them towards the answer you want. then, when you present the answers to the poll, you also cage the results in such a way to lead the audience in the way you want them to interpret the results
polling is fucking joke. all results from the left, or the right, is complete bullshit, and a waste of your time
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
If you mod me down, I shall become more powerful than you could possibly imagine.
Tin Hat Alert doesn't begin to cover this.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
In Neal Stephenson's Cryptonomicon, there's a scene early in the book where the Allies are assembling the personnel for Station X (aka Bletchely Park). Statistician, turned Nazi codebreaker Lawrence Waterhouse, points out that his Nazi counterpart Rudy von Hacklheber, would notice something was amiss with the Allied personnel changes based the statistics of people being transfered to Bletchely Park, and then quickly deduce that the Allies are attempting to break the Enigma code. To camouflage the transfers, Waterhouse suggests creating ficticious personnel and have some of them transfered to Bletchely Park as well. However the military can't just make any random fake person, the fictious people must be statisitically drawn from a distribution that when added to distribution of real Bletchely Park personnel, the combined distribution is statistically insignificant (i.e. fail to reject the null hypothesis) than any other large military base.
If Research 2000 did what is suggested, they failed to taint the polls with the right kind of fake data, just like what the novel warned about.
They did look at it critically. Research 2000 was fired by Daily Kos before anyone noted any impropriety in the figures, simply because the numbers weren't matching up with reality. Shortly after this happened, Grebner, Weissman, and Weissman approached Markos with evidence of deliberate impropriety.
Does Daily Kos have a responsibility to not promote questionable information as truth? Of course, and they've apologized for the situation. But keep in mind that this information is only coming to light because someone with sufficient statistical background took the time to pore over the data. That sort of expertise is hard to come by, which is the reason why smaller media/news outlets contract out to firms like Research 2000 in the first place!
It's only relatively recently that there's been much interest in the science of polling. Before the emergence of aggregation sites like FiveThirtyEight or Pollster.com, it was extremely rare that you'd ever see this kind of statistical analysis of polling data. The traditional method of testing a pollster's reliability was simple trial and error over a period of several elections. Really, that's *still* the primary method. If anything, Research 2000 only got scrutinized in this case because of the issues with their accuracy that led to them being dropped in the first place.
For me, it's not really a partisan issue, despite the highly politicized nature of Daily Kos. It has more to do with the size of the media outlet. I would expect a major news organization with dozens or hundreds of employees, like Fox News or MSNBC, to be able to detect problems like these very quickly. A relatively small blog with maybe a dozen part-time employees like Daily Kos, or Red State, or whatever, I'm more willing to give a pass. At least at first: I'd expect Markos to learn his lesson from this and be more proactive in ensuring that it doesn't happen again.
Sean Daugherty "I have walked in Eternity -- and Eternity weeps."
The pollster was subscribed to by DailyKos, among hundreds of other news organizations, and the results were skewed IN FAVOR OF RIGHT-WING CAUSES, not left-wing, so the assumption that DailyKos was somehow complicit in this is absolutely not true. (And I've rarely, if ever, read DailyKos, so I have no personal interest in defending them.. the headline is just grossly misleading).
That was Strategic Vision, not R2K.
(Hey, I'd be much happier if people named products with distinguishable proper names rather than generic sounding word combinations and worse yet, acronyms, so you have my sympathies for getting them mixed up.)
Someone had to do it.
Every Single Health Care Reform Idea is... ... a Republican idea.
You seem to forget that the National health reform model is modeled after the Massachusetts one, instigated by Mitt Romney, a Republican.
The Republican Party is only out for itself. If it's their idea,
Oh fuckit. I'm not writing this for the billionth time. Fuck the GOP, Fuck the Teabaggers, and Fuck you and your fucking short term memory.
--
BMO
Did you know the original Teabaggers were protesting the fact that the wealthy British lowered taxes on their own tea below the taxes on the colonial tea?
That's a pretty inaccurate depiction of the Tea Act and why the colonists opposed it. In essence, the British government was protecting it's own favored company (East India Company), in favor of other traders (and smugglers, because tea carried a hefty tax). So actually the colonists favored free trade instead of crony capitalism (or fascism, if you prefer), and when the British government tried to pass laws that provided monopolies for East India, the colonists rebelled.
I think that's a pretty good analogy with motivations of the modern-day Tea Party protesters.
"Somebody has to do something. It's just incredibly pathetic it has to be us."
--- Jerry Garcia
This is late but...
The fact is that the "tea party" and those that back it were utterly silent.
They were utterly silent when GWB enacted the PATRIOT Act and people like me are *traitors* for opposing it.
They were utterly silent when GWB suspended Habeas Corpus. Hello?
They are *utterly blind* when a Republican does something that is supposedly against their principles, but when a Democrat adopts a Republican idea, woe be unto him. He's a TRAITOR to the US.
They're just a branch of Republicanism and nothing more.
--
BMO