Slashdot Mirror
Political Polls Become Less Reliable As We Head Into 2016 Presidential Election
HughPickens.com writes: Cliff Zukin writes in the NY Times that those paying close attention to the 2016 election should exercise caution as they read the polls — election polling is in near crisis as statisticians say polls are becoming less reliable. According to Zukin, two trends are driving the increasing unreliability of election and other polling in the United States: the growth of cellphones and the decline in people willing to answer surveys. Coupled, they have made high-quality research much more expensive to do, so there is less of it. This has opened the door for less scientifically-based, less well-tested techniques.
To top it off, a perennial election polling problem, how to identify "likely voters," has become even thornier. Today, a majority of people are difficult or impossible to reach on landline phones. One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone. To complete a 1,000-person survey, it's not unusual to have to dial more than 20,000 random numbers, most of which do not go to actual working telephone numbers.
The second unsettling trend is rapidly declining response rates, reaching levels once considered unimaginable. In the late 1970s, pollsters considered an 80 percent response rate acceptable, but by 2014 the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven't figured out how to replace it," concludes Zukin. "In short, polls and pollsters are going to be less reliable. We may not even know when we're off base. What this means for 2016 is anybody's guess."
To top it off, a perennial election polling problem, how to identify "likely voters," has become even thornier. Today, a majority of people are difficult or impossible to reach on landline phones. One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone. To complete a 1,000-person survey, it's not unusual to have to dial more than 20,000 random numbers, most of which do not go to actual working telephone numbers.
The second unsettling trend is rapidly declining response rates, reaching levels once considered unimaginable. In the late 1970s, pollsters considered an 80 percent response rate acceptable, but by 2014 the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven't figured out how to replace it," concludes Zukin. "In short, polls and pollsters are going to be less reliable. We may not even know when we're off base. What this means for 2016 is anybody's guess."
Who wants to be Nate Silver will be able to make sense of the polls?
Still some interesting points, and yes we may reach a point where polls actually have no predictive value. But I doubt we've gone from "100% accurate if you know how to interpret them" to 0% in 4 years ;-)
You're forgetting about focus groups, which is where most politician's views/presentations are actually crafted. Polls are used as feedback for "how are we doing with 20 to 30 year-old Latino transvestites who self-identify as Republicans" to identify where (demographically) more advertising money needs to be spent.
first you must discredit the polls.
The opinion polling industry here in the UK got got last month's General Election badly wrong. Not only did almost all of the pre-election polls (conducted by a wide range of companies, some of whom use telephone surveys while others use an online approach) get the vote-distribution wrong, over-estimating Labour support and under-estimating Conservative support, but they also misread the mapping of those vote-shares into Parliamentary seats (which is, admittedly, not always simple in the UK's first-past-the-post electoral system).
Only the exit-poll conducted on the day of the election itself got relatively close to the actual result (and even that under-estimated the scale of the eventual Conservative victory).
There's a major industry post-mortem in progress at the moment, which is scrutinising various aspects of previous methodological orthodoxy. UKpollingreport has a fairly good write-up of the state of play here.
There's been a fair degree of political acrimony about the inaccuracy of the pre-election polling. In particular, there have been questions raised about whether inaccurate polling caused the parties or the voters to change their behaviours in a way that accurate polling (or no polling) wouldn't have. There are also some calls for the UK to follow the example of some continental European countries and ban the publication of opinion polls in the 2-3 week period before an election.
One other point worth noting is that there was one particular data-analytics organisation (sorry, can't find the link right now) which looked at the raw data from the opinion polls and made a call a few days before the election which predicted the outcome fairly accurately.
Nate Silver called it badly wrong, in this instance.
1) your friends are idiots if they let their intentions change due to what some poll says.
2) This ain't a new phenomenon, at all
3) A poll result does not necessarily mean that it matches the election result. See also "Dewey Defeats Truman"
3) Reagan won by a frickin' landslide in both elections, so it's not as if the media outlet had jumped any guns.
Quo usque tandem abutere, Nimbus, patientia nostra?