Slashdot Mirror


Daily Kos Pollster Made Up Numbers

jamie found a story up on Daily Kos revealing that the polling firm they had contracted with for 18 months, Research 2000 or R2K, apparently made up or at least manually tweaked its polling results. The blog published a preliminary report by a team of statistics gurus (Mark Grebner, Michael Weissman, and Jonathan Weissman), and it is an exemplar of clarity and concision. The team reports, "We do not know exactly how the weekly R2K results were created, but we are confident they could not accurately describe random polls." Daily Kos will be filing a lawsuit against its former pollster. "For the past year and a half, Daily Kos has been featuring weekly poll results from the Research 2000 (R2K) organization. These polls were often praised for their 'transparency,' since they included detailed cross-tabs on sub-populations and a clear description of the random dialing technique. However, on June 6, 2010, FiveThirtyEight.com rated R2K as among the least accurate pollsters in predicting election results. Daily Kos then terminated the relationship. One of us (MG) wondered if odd patterns he had noticed in R2K's reports might be connected with R2K's mediocre track record, prompting our investigation of whether the reports could represent proper random polling. ... This posting is a careful initial report of our findings, not intended to be a full formal analysis but rather to alert people not to rely on R2K's results."

38 of 546 comments (clear)

  1. To be fair... by Anonymous Coward · · Score: 3, Informative

    Nobody expects the Daily Kos to be accurate.

    It would be like trusting Ann Coulter, Rush Limbaugh, Sean Hannity, or anything ever aired on "Air America" before it went bankrupt.

    1. Re:To be fair... by RyuuzakiTetsuya · · Score: 3, Informative

      Actually, I do. And now They're suing the pants off of R2K.

      If this was the National Review Online, or Free Republic, or what have you, there would be a huge push to cover this up and blame the "liberal media"(whatever the hell THAT is) for any accusations that they did something wrong.

      --
      Non impediti ratione cogitationus.
    2. Re:To be fair... by skids · · Score: 2, Informative

      You need to travel. What passes for centrist in this country, and much of what is called liberal by know-nothings, is considered rather right wing in most of the rest of the western world.

    3. Re:To be fair... by spun · · Score: 5, Informative

      Wrong. Now that teabaggers know what the term means, they call themselves tea partiers. But back in the day, they carried teabags around and called themselves teabaggers.

      Here's an article backing up that fact, but I warn you, it is from that den of liberal iniquity, Billy Buckley's The National Review, so take it with the grain of salt that any reading of The National Review requires.

      http://nrd.nationalreview.com/article/?q=Mjk1YmRjNzIxNmUwMTI0ZWYxZWU4OWU2MzFiOWJmNDE=

      --
      - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
    4. Re:To be fair... by NotBornYesterday · · Score: 3, Informative

      The snarky ad hominem bit was "teabagger". And don't pretend you didn't mean it that way.

      --
      I prefer rogues to imbeciles because they sometimes take a rest.
    5. Re:To be fair... by spun · · Score: 2, Informative

      Wrong.

      The first big day for this movement was Tax Day, April 15. And organizers had a gimmick. They asked people to send a tea bag to the Oval Office. One of the exhortations was “Tea Bag the Fools in D.C.” A protester was spotted with a sign saying, “Tea Bag the Liberal Dems Before They Tea Bag You.” So, conservatives started it: started with this terminology. But others ran with it and ran with it.

      --
      http://nrd.nationalreview.com/article/?q=Mjk1YmRjNzIxNmUwMTI0ZWYxZWU4OWU2MzFiOWJmNDE=

      --
      - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
    6. Re:To be fair... by Curunir_wolf · · Score: 5, Informative

      Nope, that proves nothing other than some people are as ignorant as yourself. That article was written in December, 2009. And the author apparently didn't know anything about the Tea Parties that had been happening for almost three years - he seems under the (mistaken, or intentionally misleading) assumption it had something to do with Obama's election.

      Here's some insight from some of the progenitor tea parties.

      --
      "Somebody has to do something. It's just incredibly pathetic it has to be us."
      --- Jerry Garcia
    7. Re:To be fair... by Curunir_wolf · · Score: 2, Informative

      Nice (second attempt) at this revisionist history, but you are the one that has it wrong.

      Nope, that proves nothing other than some people are as ignorant as yourself. That article was written in December, 2009. And the author apparently didn't know anything about the Tea Parties that had been happening for almost three years - he seems under the (mistaken, or intentionally misleading) assumption it had something to do with Obama's election.

      Here's an article, and a video from an early tea party where nobody called themselves "teabaggers" (yea, it came from the snarky left, apparently in fear of a grass-roots conservative movement).

      --
      "Somebody has to do something. It's just incredibly pathetic it has to be us."
      --- Jerry Garcia
    8. Re:To be fair... by NotBornYesterday · · Score: 3, Informative

      Like I said, it has no place in polite conversation, regardless of who uses it. Plus, many who are being tarred with this brush had no part in its origins.

      And none of that changes my original point, which was that it is used as a pejorative term to attack people rather than engage in debate, and therefore is usually used by people whose ability to engage in an interesting discussion is less developed than their desire to mock those who think differently.

      --
      I prefer rogues to imbeciles because they sometimes take a rest.
    9. Re:To be fair... by DJRumpy · · Score: 4, Informative

      Actually no. They carried the very signs that started all of this:

      http://washingtonindependent.com/69660/correcting-jay-nordlinger

      In January of 09, they had a Facebook page that had some back and forth discussion about the 'alternate' meaning of teabag with some surprised disdain when they were informed as to what the term meant. They were apparently unaware at that point.

      This is from the rally in DC on April 15th of 2009:
      http://washingtonindependent.com/31868/scenes-from-the-new-american-tea-party

      One final little tidbit...the debate by conservatives as to whether or not to wear the title with pride ;)

      http://tpmdc.talkingpointsmemo.com/2009/12/to-teabag-or-not-thats-still-the-question-for-conservatives.php

    10. Re:To be fair... by 644bd346996 · · Score: 2, Informative

      1913 isn't really a fair data point to use, considering that that was the year that federal income taxes were first explicitly allowed by the Constitution. Insight can be had by realizing that by the time that federal income tax was 4 years old, the top rate had grown to 67% (though most of this was to fund WW1).

      However, any discussion of top marginal tax rates is incomplete and even disingenuous without considering how much you had to earn in order to qualify for that top bracket. A graph like the one at http://www.visualizingeconomics.com/2007/11/03/nytimes-historical-tax-rates-by-income-group/ is necessary to accurately convey the change in tax structure over time. The super-rich elite truly have had it easier in recent decades, but during the 90's most of the population was subject to more progressive taxation than during the 60's.

    11. Re:To be fair... by NotBornYesterday · · Score: 1, Informative

      1) monitor product safety, especially food and drugs, 2) protect the environment, and 3) fund scientific research for which there is no immediate commercial application.

      We would have lots of money to do that if we (and by "we", I don't actually mean "you and I", I mean the Republicrats) weren't spending it foolishly elsewhere. But look at where we are now. We are spending so much so-called stimulus money that even the Europeans are looking at us and saying "Uh, no thanks, we'd rather cut back our spending than try to keep up with you". And surprise, surprise, the Democrats are getting ready to raise taxes again. Not just on those rich folks they demonized during the elections either; they are coming after your paycheck and mine.

      #4) prosecute businesses that engage in anti-competitive monopolistic behavior.

      Hell yeah. I don't like big government interfering in business, but a healthy free-market system requires a free market, which a monopoly certainly isn't. If there is need for a limited government role there, fine.

      --
      I prefer rogues to imbeciles because they sometimes take a rest.
  2. Slightly misleading headline? by Pojut · · Score: 3, Informative

    Those who aren't used to phrases used with "political" centric organizations might mistake the title as saying someone who is on Daily Kos' payroll flubbed the numbers, rather than a company working on contract with them.

    1. Re:Slightly misleading headline? by Ocker3 · · Score: 2, Informative

      even simply adding an apostrophe to make it "Daily Kos' pollster made up numbers" would be more informative

    2. Re:Slightly misleading headline? by blueg3 · · Score: 2, Informative

      Unless Kos is plural, since it's not the name of an ancient person, they'll need to add an apostrophe and an s. "Daily Kos's Pollster Made Up Numbers".

    3. Re:Slightly misleading headline? by Volante3192 · · Score: 2, Informative

      It's technically correct (the best kind of correct) that you can show possession with simply an apostrophe for singular nouns ending in an 'ess' sound.

      The trick is you have to be consistent about it. (You can't start with "Daily Kos' pollster" and later use "Daily Kos's editor".)

      It's more of a guideline than a rule to use the succeeding 's.'

      However, I will say that leaving the 's' off would likely be a depreciated style if this was a standards documentation.

  3. Polling by Mongoose+Disciple · · Score: 2, Informative

    For me, the surprising part of this isn't so much that R2K made up poll results, but that the results actually were noticably less accurate than traditional polling, which I like to think of as representing a broad cross-section of people who still have landlines with no caller ID for some reason (or are desperate enough to talk to another human being that they'll answer their landline anyway).

  4. R2K not alone in this. by Zephyr14z · · Score: 5, Informative

    I used to work a different major polling company, and I can assure you R2K is not alone in just making up numbers. Easily 80% of surveys that went through my region were completely falsified, and the remaining 20% rarely matched the demographic they were supposed to be answering for. Survey administrators have quotas, and then get paid extra for additional surveys past that, but there is basically nothing done to verify any of the surveys turned in, and everybody in the company knows it. Don't always trust what you read, especially not statistics.

  5. misleading headline... by emagery · · Score: 4, Informative

    For headline skimmers, this post would produced a completely inaccurate sense of what the article was all about... at length, the D.KOS are the ones who found out about this and are doing something something about it. That's good... but if you just read the headline, you'd come away thinking that D.KOS were the culprits instead.

    1. Re:misleading headline... by DerekLyons · · Score: 1, Informative

      That's pretty much not what happened. The Daily Kos published the poll results without review or oversight - until a third party found the pollster to be unreliable and a fourth party found the particular polls published by the Daily Kos to be flawed.

      So yeah, I'd lump the Daily Kos in among the culprits for failing to properly review the material they were publishing.

  6. Re:Give them credit. by dkleinsc · · Score: 3, Informative

    Apparently the situation with Rasmussen is complicated, but this seems to be a fairly decent starting place (that's not just some activist blogger).

    --
    I am officially gone from /. Long live http://www.soylentnews.com/
  7. You Are Not a Republican by Gary+W.+Longsine · · Score: 4, Informative
    You're also lazy, and ill informed. You could have spent a fraction of a second (0.15 seconds) with Google to find about 3,860,000 results for the search term "Rasmussen bias" to discover that, yes, in fact, there is some discussion of this point.

    Nate Silver on Possible Biasin Rasmussen Reports
    "What Rasmussen has had is a "house effect". So far in the 2010 cycle, their polling has consistently and predictably shown better results for Republican candidates than other polling firms have. But such house effects can emerge from legitimate differences of opinion about how to model the electorate. And ultimately, these differences of opinion will be tested -- based on what happens next November. If Rasmussen's opinion turns out to be wildly inaccurate, that will impeach their credibility, and believe me, we will point that out. Likewise, if they turn out to be right when most other pollsters are wrong, we will point that out too."

    --
    If you mod me down, I shall become more powerful than you could possibly imagine.
    1. Re:You Are Not a Republican by oatworm · · Score: 2, Informative

      Similarly, I slept with your mom returns 2.1M results. Also, 910k results say your mom is a whore.

      Seriously though, yeah, Google Confirmation Bias is an incredibly fun game to play.

  8. Re:Mark Twain said it best by Anonymous Coward · · Score: 1, Informative

    For actual details on how you can tell that stuff was just made up, start reading here: http://www.fivethirtyeight.com/2010/06/breaking-daily-kos-to-sue-research-2000.html

  9. Re:Give them credit. by Attila+Dimedici · · Score: 4, Informative

    The quote you gave merely says that the results that Rasmussen is getting at this time show a public more greatly favoring Republican positions not that Rasmussen is fudging the numbers to get those results. The rest of the article can be dismissed since the source for the article clearly favors Democratic policy over Republican

    --
    The truth is that all men having power ought to be mistrusted. James Madison
  10. Remember: by Locke2005 · · Score: 2, Informative

    If you place a statistician's head in ice and his feet in boiling water, then on the average he is quite comfortable!

    --
    I've abandoned my search for truth; now I'm just looking for some useful delusions.
  11. Re:Commence Right-Wing Yank-Fest by Volante3192 · · Score: 2, Informative
  12. Re:the truth is, polling sucks by Entropius · · Score: 2, Informative

    Daily Kos wasn't trying to manipulate anything -- notice that they fired R2K once fivethirtyeight's statistics showed them to be least accurate at predicting election results, long before there was any evidence of fraud?

  13. Re:Give them credit. by Orange+Crush · · Score: 2, Informative

    Actually, Kos had already fired R2K before the study was performed. Evidence of fraud was only recently discovered.

  14. Re:I am not sure who those "teabaggers" are... by Uberbah · · Score: 2, Informative

    Too damn bad:

    The current Tea Party movement initially came up with the 'clever' idea of sending Tea bags to members of Congress. It was in their initial rallys that they started referring to this act of sending their representatives boxes of tea bags as 'TEABAGGING":

    Now this may be a generational thing but somebody should have told these people that the term was already in wide use as a term for performing oral sex on a man. Now you got people like Tucker Carlson crying: "Stop saying Teabagger". I got news for all of you Teabaggers that are opposed to the use of the word Teabagging when they are out doing their Teabagging protest; You really shot yourself in the foot (or other appendage) when you started this mess. But then what do you expect from the extremist that our on a highly successful mission to divide and destroy the once Grand Old Party, GOP. Oh yeah, and they really didn't mean to destroy the country their supposed to be saving either. Collateral Damage its called!

    But what do you expect from people who start holding anti-tax protests after Obama signed the largest middle class tax cut in history? What do you expect from the kind of geniuses that hold up signs saying "Keep Your Government Hands off My Medicare".

  15. Re:I am not sure who those "teabaggers" are... by spun · · Score: 2, Informative

    I saw participants referring to themselves as teabaggers in the beginning, and the idiotic right wing commentators at Faux News picked it up and ran with it before they figured out what it meant. Which is hilarious because the initial Tea Party events were sponsored by Faux News.

    --
    - None can love freedom heartily, but good men; the rest love not freedom, but license. -- John Milton
  16. Re:Mark Twain said it best by jkauzlar · · Score: 3, Informative
    The headline on this article was stupidly misleading. Months ago, if not over a year, Nate Silver at fivethirtyeight.com called out R2K for just this one thing. You may remember Silver's interesting observation that the least significant digits in the polling results did not follow a normal random distribution. For example there may have been too many .9's in the results (58.9 or 63.9, etc) while there were few instances of other digits.

    The pollster was subscribed to by DailyKos, among hundreds of other news organizations, and the results were skewed IN FAVOR OF RIGHT-WING CAUSES, not left-wing, so the assumption that DailyKos was somehow complicit in this is absolutely not true. (And I've rarely, if ever, read DailyKos, so I have no personal interest in defending them.. the headline is just grossly misleading).

  17. Re:Mark Twain said it best by skids · · Score: 3, Informative

    That was Strategic Vision, not R2K.

    (Hey, I'd be much happier if people named products with distinguishable proper names rather than generic sounding word combinations and worse yet, acronyms, so you have my sympathies for getting them mixed up.)

  18. Re:Are you for some reason surprised? by bonch · · Score: 2, Informative

    But my point is that there aren't equal numbers. Slashdot has always leaned one way--left. In this one case where a left-wing site is the topic of discussion, the accusation is being made that Slashdot is a conservative site.

    By the way, the Kos supporters with mod points are out in full force abusing the -1 Overrated moderation.

  19. Re:I am not sure who those "teabaggers" are... by Curunir_wolf · · Score: 3, Informative

    Did you know the original Teabaggers were protesting the fact that the wealthy British lowered taxes on their own tea below the taxes on the colonial tea?

    That's a pretty inaccurate depiction of the Tea Act and why the colonists opposed it. In essence, the British government was protecting it's own favored company (East India Company), in favor of other traders (and smugglers, because tea carried a hefty tax). So actually the colonists favored free trade instead of crony capitalism (or fascism, if you prefer), and when the British government tried to pass laws that provided monopolies for East India, the colonists rebelled.

    I think that's a pretty good analogy with motivations of the modern-day Tea Party protesters.

    --
    "Somebody has to do something. It's just incredibly pathetic it has to be us."
    --- Jerry Garcia
  20. Re:I am not sure who those "teabaggers" are... by Apple+Acolyte · · Score: 2, Informative

    Sure bmo, like most Republicans supported Romney Care. Romney's probably taken himself out of contention as a future Republican nominee because of Romney Care. Are you claiming the Tea Parties supported Romney Care or Obamacare? Was this post from you even for real, or was it a joke?

    --
    Part of the hardcore faithful who believed in Apple long before it was cool again to do so
  21. Re:I am not sure who those "teabaggers" are... by OeLeWaPpErKe · · Score: 2, Informative

    By your standards the democrats are in favor of restarting the holocaust, using hamas as the new nazis. After all, some are (just read dailykos for a bit).

    And generally, none of them say anything when hamas makes statements about restarting the holocaust. And everytime anyone suggests that maybe hamas should be taught a lesson "democrats" (let's throw everyone in one basket like you so seem to enjoy) rant on and on about how "victimized" those poor genocidal maniacs are.

    And let's not pretend that the lunatics of, say dailykos or democratic underground, don't know perfectly well that what they're saying about gazans is simply a lie.