Slashdot Mirror
Daily Kos Pollster Made Up Numbers
jamie found a story up on Daily Kos revealing that the polling firm they had contracted with for 18 months, Research 2000 or R2K, apparently made up or at least manually tweaked its polling results. The blog published a preliminary report by a team of statistics gurus (Mark Grebner, Michael Weissman, and Jonathan Weissman), and it is an exemplar of clarity and concision. The team reports, "We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls." Daily Kos will be filing a lawsuit against its former pollster. "For the past year and a half, Daily Kos has been featuring weekly poll results from the Research 2000 (R2K) organization. These polls were often praised for their 'transparency,' since they included detailed cross-tabs on sub-populations and a clear description of the random dialing technique. However, on June 6, 2010, FiveThirtyEight.com rated R2K as among the least accurate pollsters in predicting election results. Daily Kos then terminated the relationship. One of us (MG) wondered if odd patterns he had noticed in R2K's reports might be connected with R2K's mediocre track record, prompting our investigation of whether the reports could represent proper random polling. ... This posting is a careful initial report of our findings, not intended to be a full formal analysis but rather to alert people not to rely on R2K's results."
Nobody expects the Daily Kos to be accurate.
It would be like trusting Ann Coulter, Rush Limbaugh, Sean Hannity, or anything ever aired on "Air America" before it went bankrupt.
Those who aren't used to phrases used with "political" centric organizations might mistake the title as saying someone who is on Daily Kos' payroll flubbed the numbers, rather than a company working on contract with them.
Living With a Nerd
For me, the surprising part of this isn't so much that R2K made up poll results, but that the results actually were noticably less accurate than traditional polling, which I like to think of as representing a broad cross-section of people who still have landlines with no caller ID for some reason (or are desperate enough to talk to another human being that they'll answer their landline anyway).
In this case it was Lies, Damned Lies, and "stuff we made up".
Whether or not there is some sort of god, I'm not supposed to say/god is a word and the argument ends there-Smog
Unlike the many Republican outfits which used partly- or wholly-fabricated polls by Strategic Vision, or the many media outlets which continue to use the horribly flawed Rasmussen polls to create eye-catching headlines, Kos immediately dumped the pollster, did an investigation, owned up to the errors publicly, and is now pursuing legal recourse.
This is exactly how you would expect an honest media organization (if one with a considerable political agenda) to behave. Too bad the mainstream media and those on the other side of the aisle don't seem to want to do the same.
You'd think they would spend Soros money more carefully.
Professional Politicians are not the solution, they ARE the problem.
I used to work a different major polling company, and I can assure you R2K is not alone in just making up numbers. Easily 80% of surveys that went through my region were completely falsified, and the remaining 20% rarely matched the demographic they were supposed to be answering for. Survey administrators have quotas, and then get paid extra for additional surveys past that, but there is basically nothing done to verify any of the surveys turned in, and everybody in the company knows it. Don't always trust what you read, especially not statistics.
Everyone knows that 78.49% of statistics are made up.
And that the other 23% are erroneous.
They say a little knowledge is a dangerous thing, but it's not one half so bad as a lot of ignorance. - Terry Pratchett
For headline skimmers, this post would produced a completely inaccurate sense of what the article was all about... at length, the D.KOS are the ones who found out about this and are doing something something about it. That's good... but if you just read the headline, you'd come away thinking that D.KOS were the culprits instead.
Comment removed based on user account deletion
of course this will turn into a "bash the left" and a "bash the right" thread. when ideology isn't the point. polling is the idiocy in question
and the guy who manipulated the numbers is clearly an amateur. the way you do proper poll manipulation is LOAD THE QUESTION. you poll people with a question with the proper turn of phrase to lead them towards the answer you want. then, when you present the answers to the poll, you also cage the results in such a way to lead the audience in the way you want them to interpret the results
polling is fucking joke. all results from the left, or the right, is complete bullshit, and a waste of your time
intellectual property law is philosophically incoherent. it is your moral duty to ignore it or sabotage it
If you mod me down, I shall become more powerful than you could possibly imagine.
If you place a statistician's head in ice and his feet in boiling water, then on the average he is quite comfortable!
I've abandoned my search for truth; now I'm just looking for some useful delusions.
And when you look at it from another perspective - you will probably exclude the nerd section of the population who never answers calls from 800-numbers (or other known junk callers) by using technology to divert the calls into a tarpit or something.
At least the open source telephony switch Asterisk do have a blacklist function where blacklisted numbers can be stored and used to perform a response like "The number you have dialed is not in use".
I do run that feature myself - and it's a lot more effective than using those "do not call" registries.
If builders built buildings the way programmers wrote programs, then the first woodpecker would destroy civilization.
Tin Hat Alert doesn't begin to cover this.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
In Neal Stephenson's Cryptonomicon, there's a scene early in the book where the Allies are assembling the personnel for Station X (aka Bletchely Park). Statistician, turned Nazi codebreaker Lawrence Waterhouse, points out that his Nazi counterpart Rudy von Hacklheber, would notice something was amiss with the Allied personnel changes based the statistics of people being transfered to Bletchely Park, and then quickly deduce that the Allies are attempting to break the Enigma code. To camouflage the transfers, Waterhouse suggests creating ficticious personnel and have some of them transfered to Bletchely Park as well. However the military can't just make any random fake person, the fictious people must be statisitically drawn from a distribution that when added to distribution of real Bletchely Park personnel, the combined distribution is statistically insignificant (i.e. fail to reject the null hypothesis) than any other large military base.
If Research 2000 did what is suggested, they failed to taint the polls with the right kind of fake data, just like what the novel warned about.
I know, I know, sarcasm, but I just couldn't help think of this:
http://livefeed.hollywoodreporter.com/2009/12/fox-news-120-have-opinion-on-climate-research-pic.html
You've never heard of exit polls? They're pretty much dead on in predicting vote results if done properly. There was some controversy in Bush vs. Kerry about them, it's interesting reading.
Intron: the portion of DNA which expresses nothing useful.
How? Lots of hand-waving, misdirection, and questionable assumptions. The whole research industry is in total denial about it, because there aren't any good alternatives yet.
They did look at it critically. Research 2000 was fired by Daily Kos before anyone noted any impropriety in the figures, simply because the numbers weren't matching up with reality. Shortly after this happened, Grebner, Weissman, and Weissman approached Markos with evidence of deliberate impropriety.
Does Daily Kos have a responsibility to not promote questionable information as truth? Of course, and they've apologized for the situation. But keep in mind that this information is only coming to light because someone with sufficient statistical background took the time to pore over the data. That sort of expertise is hard to come by, which is the reason why smaller media/news outlets contract out to firms like Research 2000 in the first place!
It's only relatively recently that there's been much interest in the science of polling. Before the emergence of aggregation sites like FiveThirtyEight or Pollster.com, it was extremely rare that you'd ever see this kind of statistical analysis of polling data. The traditional method of testing a pollster's reliability was simple trial and error over a period of several elections. Really, that's *still* the primary method. If anything, Research 2000 only got scrutinized in this case because of the issues with their accuracy that led to them being dropped in the first place.
For me, it's not really a partisan issue, despite the highly politicized nature of Daily Kos. It has more to do with the size of the media outlet. I would expect a major news organization with dozens or hundreds of employees, like Fox News or MSNBC, to be able to detect problems like these very quickly. A relatively small blog with maybe a dozen part-time employees like Daily Kos, or Red State, or whatever, I'm more willing to give a pass. At least at first: I'd expect Markos to learn his lesson from this and be more proactive in ensuring that it doesn't happen again.
Sean Daugherty "I have walked in Eternity -- and Eternity weeps."
Too damn bad:
But what do you expect from people who start holding anti-tax protests after Obama signed the largest middle class tax cut in history? What do you expect from the kind of geniuses that hold up signs saying "Keep Your Government Hands off My Medicare".
I saw participants referring to themselves as teabaggers in the beginning, and the idiotic right wing commentators at Faux News picked it up and ran with it before they figured out what it meant. Which is hilarious because the initial Tea Party events were sponsored by Faux News.
- None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
The pollster was subscribed to by DailyKos, among hundreds of other news organizations, and the results were skewed IN FAVOR OF RIGHT-WING CAUSES, not left-wing, so the assumption that DailyKos was somehow complicit in this is absolutely not true. (And I've rarely, if ever, read DailyKos, so I have no personal interest in defending them.. the headline is just grossly misleading).
That was Strategic Vision, not R2K.
(Hey, I'd be much happier if people named products with distinguishable proper names rather than generic sounding word combinations and worse yet, acronyms, so you have my sympathies for getting them mixed up.)
Someone had to do it.
When you are attempting to do as little work as possible and still get the million dollars a year kos spends on polls, mocking with the data in such a way that nothing amiss can be detected is rather counter productive: You might as well do the polls right anyway.
But my point is that there aren't equal numbers. Slashdot has always leaned one way--left. In this one case where a left-wing site is the topic of discussion, the accusation is being made that Slashdot is a conservative site.
By the way, the Kos supporters with mod points are out in full force abusing the -1 Overrated moderation.
Every Single Health Care Reform Idea is... ... a Republican idea.
You seem to forget that the National health reform model is modeled after the Massachusetts one, instigated by Mitt Romney, a Republican.
The Republican Party is only out for itself. If it's their idea,
Oh fuckit. I'm not writing this for the billionth time. Fuck the GOP, Fuck the Teabaggers, and Fuck you and your fucking short term memory.
--
BMO
Did you know the original Teabaggers were protesting the fact that the wealthy British lowered taxes on their own tea below the taxes on the colonial tea?
That's a pretty inaccurate depiction of the Tea Act and why the colonists opposed it. In essence, the British government was protecting it's own favored company (East India Company), in favor of other traders (and smugglers, because tea carried a hefty tax). So actually the colonists favored free trade instead of crony capitalism (or fascism, if you prefer), and when the British government tried to pass laws that provided monopolies for East India, the colonists rebelled.
I think that's a pretty good analogy with motivations of the modern-day Tea Party protesters.
"Somebody has to do something. It's just incredibly pathetic it has to be us."
--- Jerry Garcia
Sure bmo, like most Republicans supported Romney Care. Romney's probably taken himself out of contention as a future Republican nominee because of Romney Care. Are you claiming the Tea Parties supported Romney Care or Obamacare? Was this post from you even for real, or was it a joke?
Part of the hardcore faithful who believed in Apple long before it was cool again to do so
This is late but...
The fact is that the "tea party" and those that back it were utterly silent.
They were utterly silent when GWB enacted the PATRIOT Act and people like me are *traitors* for opposing it.
They were utterly silent when GWB suspended Habeas Corpus. Hello?
They are *utterly blind* when a Republican does something that is supposedly against their principles, but when a Democrat adopts a Republican idea, woe be unto him. He's a TRAITOR to the US.
They're just a branch of Republicanism and nothing more.
--
BMO
By your standards the democrats are in favor of restarting the holocaust, using hamas as the new nazis. After all, some are (just read dailykos for a bit).
And generally, none of them say anything when hamas makes statements about restarting the holocaust. And everytime anyone suggests that maybe hamas should be taught a lesson "democrats" (let's throw everyone in one basket like you so seem to enjoy) rant on and on about how "victimized" those poor genocidal maniacs are.
And let's not pretend that the lunatics of, say dailykos or democratic underground, don't know perfectly well that what they're saying about gazans is simply a lie.