Slashdot Mirror
Daily Kos Pollster Made Up Numbers
jamie found a story up on Daily Kos revealing that the polling firm they had contracted with for 18 months, Research 2000 or R2K, apparently made up or at least manually tweaked its polling results. The blog published a preliminary report by a team of statistics gurus (Mark Grebner, Michael Weissman, and Jonathan Weissman), and it is an exemplar of clarity and concision. The team reports, "We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls." Daily Kos will be filing a lawsuit against its former pollster. "For the past year and a half, Daily Kos has been featuring weekly poll results from the Research 2000 (R2K) organization. These polls were often praised for their 'transparency,' since they included detailed cross-tabs on sub-populations and a clear description of the random dialing technique. However, on June 6, 2010, FiveThirtyEight.com rated R2K as among the least accurate pollsters in predicting election results. Daily Kos then terminated the relationship. One of us (MG) wondered if odd patterns he had noticed in R2K's reports might be connected with R2K's mediocre track record, prompting our investigation of whether the reports could represent proper random polling. ... This posting is a careful initial report of our findings, not intended to be a full formal analysis but rather to alert people not to rely on R2K's results."
Unlike the many Republican outfits which used partly- or wholly-fabricated polls by Strategic Vision, or the many media outlets which continue to use the horribly flawed Rasmussen polls to create eye-catching headlines, Kos immediately dumped the pollster, did an investigation, owned up to the errors publicly, and is now pursuing legal recourse.
This is exactly how you would expect an honest media organization (if one with a considerable political agenda) to behave. Too bad the mainstream media and those on the other side of the aisle don't seem to want to do the same.
I used to work a different major polling company, and I can assure you R2K is not alone in just making up numbers. Easily 80% of surveys that went through my region were completely falsified, and the remaining 20% rarely matched the demographic they were supposed to be answering for. Survey administrators have quotas, and then get paid extra for additional surveys past that, but there is basically nothing done to verify any of the surveys turned in, and everybody in the company knows it. Don't always trust what you read, especially not statistics.
For headline skimmers, this post would produced a completely inaccurate sense of what the article was all about... at length, the D.KOS are the ones who found out about this and are doing something something about it. That's good... but if you just read the headline, you'd come away thinking that D.KOS were the culprits instead.
Comment removed based on user account deletion
Perhaps you misread the summary? The Daily Kos is not at fault here.
of course this will turn into a "bash the left" and a "bash the right" thread. when ideology isn't the point. polling is the idiocy in question
and the guy who manipulated the numbers is clearly an amateur. the way you do proper poll manipulation is LOAD THE QUESTION. you poll people with a question with the proper turn of phrase to lead them towards the answer you want. then, when you present the answers to the poll, you also cage the results in such a way to lead the audience in the way you want them to interpret the results
polling is fucking joke. all results from the left, or the right, is complete bullshit, and a waste of your time
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
Somebody really fucked up the metamoderation. I used to metamoderate all the time when you had to rate moderations as fair, unfair, or neutral. I thought that worked pretty well, but now metamoderation is an up or down vote, and worst of all, by doing so you have tagged the post and it shows up on your damn profile page! I only made that mistake once, and I'm not metamoderating again until that goes away. I don't want my profile page cluttered with random posts from other people.
I support the Slashcott and will not be reading or commenting from 2/10/14 to 2/17/14. Beta is steaming pile of dog shit
If you mod me down, I shall become more powerful than you could possibly imagine.
In Neal Stephenson's Cryptonomicon, there's a scene early in the book where the Allies are assembling the personnel for Station X (aka Bletchely Park). Statistician, turned Nazi codebreaker Lawrence Waterhouse, points out that his Nazi counterpart Rudy von Hacklheber, would notice something was amiss with the Allied personnel changes based the statistics of people being transfered to Bletchely Park, and then quickly deduce that the Allies are attempting to break the Enigma code. To camouflage the transfers, Waterhouse suggests creating ficticious personnel and have some of them transfered to Bletchely Park as well. However the military can't just make any random fake person, the fictious people must be statisitically drawn from a distribution that when added to distribution of real Bletchely Park personnel, the combined distribution is statistically insignificant (i.e. fail to reject the null hypothesis) than any other large military base.
If Research 2000 did what is suggested, they failed to taint the polls with the right kind of fake data, just like what the novel warned about.
Nobody expects the Daily Kos to be accurate. It would be like trusting Ann Coulter, Rush Limbaugh, Sean Hannity, or anything ever aired on "Air America" before it went bankrupt.
On matters of fact, they're pretty scrupulous, especially when it comes to owning up to their own mistakes, like hiring R2K.
On matters of opinion and ideology, well, it's a political blog. What exactly is an "accurate" opinion?
Proud member of the Weirdo-American community.
On matters of opinion and ideology, well, it's a political blog. What exactly is an "accurate" opinion?
"Mine."
Olbermann continued to close his show with the number of days since "Mission Accomplished" right up until a month or so ago, when he switched to the number of days since the deepwater horizon leak started. Much of the left does not pull punches against Obama for taking his time extracting us from these debacles.
The far left idealists can get quite heated against Obama. Me, all I have to do is imagine how McCain would have responded, then after I've wiped off all the cold sweat and stopped gritting my teeth, I have no regrets about 08.
Someone had to do it.
They explain the criteria by which they selected R2K, and it seemed fine. (RTFA)
That they caught R2K at this, and were willing to expose it, while other polls have also exhibited some of these patterns and continued to be used by their clients, says more good things about dkos than bad.
Someone had to do it.
They did look at it critically. Research 2000 was fired by Daily Kos before anyone noted any impropriety in the figures, simply because the numbers weren't matching up with reality. Shortly after this happened, Grebner, Weissman, and Weissman approached Markos with evidence of deliberate impropriety.
Does Daily Kos have a responsibility to not promote questionable information as truth? Of course, and they've apologized for the situation. But keep in mind that this information is only coming to light because someone with sufficient statistical background took the time to pore over the data. That sort of expertise is hard to come by, which is the reason why smaller media/news outlets contract out to firms like Research 2000 in the first place!
It's only relatively recently that there's been much interest in the science of polling. Before the emergence of aggregation sites like FiveThirtyEight or Pollster.com, it was extremely rare that you'd ever see this kind of statistical analysis of polling data. The traditional method of testing a pollster's reliability was simple trial and error over a period of several elections. Really, that's *still* the primary method. If anything, Research 2000 only got scrutinized in this case because of the issues with their accuracy that led to them being dropped in the first place.
For me, it's not really a partisan issue, despite the highly politicized nature of Daily Kos. It has more to do with the size of the media outlet. I would expect a major news organization with dozens or hundreds of employees, like Fox News or MSNBC, to be able to detect problems like these very quickly. A relatively small blog with maybe a dozen part-time employees like Daily Kos, or Red State, or whatever, I'm more willing to give a pass. At least at first: I'd expect Markos to learn his lesson from this and be more proactive in ensuring that it doesn't happen again.
Sean Daugherty "I have walked in Eternity -- and Eternity weeps."
To an educated mind, snarky ad-hominem attacks do more to discredit you than your opponent.
I prefer rogues to imbeciles because they sometimes take a rest.
Wrong. Now that teabaggers know what the term means, they call themselves tea partiers. But back in the day, they carried teabags around and called themselves teabaggers.
Here's an article backing up that fact, but I warn you, it is from that den of liberal iniquity, Billy Buckley's The National Review, so take it with the grain of salt that any reading of The National Review requires.
http://nrd.nationalreview.com/article/?q=Mjk1YmRjNzIxNmUwMTI0ZWYxZWU4OWU2MzFiOWJmNDE=
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
Haha. You guys still haven't figured out that "progressive" is a term we've been using for a deviant sexual act. LOL! That's why we guffaw when you call yourself a "progressive". I mean, the only thing we can thing to ourselves is, "well, yea, somebody tried to progressivize me in college, but I just wasn't drunk and horny enough to get into it."
"Somebody has to do something. It's just incredibly pathetic it has to be us."
--- Jerry Garcia
Every Single Health Care Reform Idea is... ... a Republican idea.
You seem to forget that the National health reform model is modeled after the Massachusetts one, instigated by Mitt Romney, a Republican.
The Republican Party is only out for itself. If it's their idea,
Oh fuckit. I'm not writing this for the billionth time. Fuck the GOP, Fuck the Teabaggers, and Fuck you and your fucking short term memory.
--
BMO
Nope, that proves nothing other than some people are as ignorant as yourself. That article was written in December, 2009. And the author apparently didn't know anything about the Tea Parties that had been happening for almost three years - he seems under the (mistaken, or intentionally misleading) assumption it had something to do with Obama's election.
Here's some insight from some of the progenitor tea parties.
"Somebody has to do something. It's just incredibly pathetic it has to be us."
--- Jerry Garcia
Actually no. They carried the very signs that started all of this:
http://washingtonindependent.com/69660/correcting-jay-nordlinger
In January of 09, they had a Facebook page that had some back and forth discussion about the 'alternate' meaning of teabag with some surprised disdain when they were informed as to what the term meant. They were apparently unaware at that point.
This is from the rally in DC on April 15th of 2009:
http://washingtonindependent.com/31868/scenes-from-the-new-american-tea-party
One final little tidbit...the debate by conservatives as to whether or not to wear the title with pride ;)
http://tpmdc.talkingpointsmemo.com/2009/12/to-teabag-or-not-thats-still-the-question-for-conservatives.php