Slashdot Mirror
Political Polls Become Less Reliable As We Head Into 2016 Presidential Election
HughPickens.com writes: Cliff Zukin writes in the NY Times that those paying close attention to the 2016 election should exercise caution as they read the polls — election polling is in near crisis as statisticians say polls are becoming less reliable. According to Zukin, two trends are driving the increasing unreliability of election and other polling in the United States: the growth of cellphones and the decline in people willing to answer surveys. Coupled, they have made high-quality research much more expensive to do, so there is less of it. This has opened the door for less scientifically-based, less well-tested techniques.
To top it off, a perennial election polling problem, how to identify "likely voters," has become even thornier. Today, a majority of people are difficult or impossible to reach on landline phones. One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone. To complete a 1,000-person survey, it's not unusual to have to dial more than 20,000 random numbers, most of which do not go to actual working telephone numbers.
The second unsettling trend is rapidly declining response rates, reaching levels once considered unimaginable. In the late 1970s, pollsters considered an 80 percent response rate acceptable, but by 2014 the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven't figured out how to replace it," concludes Zukin. "In short, polls and pollsters are going to be less reliable. We may not even know when we're off base. What this means for 2016 is anybody's guess."
To top it off, a perennial election polling problem, how to identify "likely voters," has become even thornier. Today, a majority of people are difficult or impossible to reach on landline phones. One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone. To complete a 1,000-person survey, it's not unusual to have to dial more than 20,000 random numbers, most of which do not go to actual working telephone numbers.
The second unsettling trend is rapidly declining response rates, reaching levels once considered unimaginable. In the late 1970s, pollsters considered an 80 percent response rate acceptable, but by 2014 the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven't figured out how to replace it," concludes Zukin. "In short, polls and pollsters are going to be less reliable. We may not even know when we're off base. What this means for 2016 is anybody's guess."
Who wants to be Nate Silver will be able to make sense of the polls?
Still some interesting points, and yes we may reach a point where polls actually have no predictive value. But I doubt we've gone from "100% accurate if you know how to interpret them" to 0% in 4 years ;-)
What do you expect. Figures don't lie, but liars can figure.
INSERT INTO comment VALUE('Doh!') WHERE user='you';
One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone.
I know some people that the pollers could outsource to that have seemed to have found a very easy workaround to this problem.
"Hi, this is Rachel from polling services....."
The only thing necessary for evil to triumph is for it to be pitted against a slightly greater evil
... it means you wait until the votes are counted to declare a winner instead of when the press tells you who the winner is.
Post Citizens United we're going to get the best government that money can buy.
One reason why polling companies can't get usable info is that end users tend to be constantly barraged by robocalls, be it the GE security system, "polls" which actually turn out to be scammy sales pitches, or many other types of scams.
Because of this, people either use apps like Mr. Number which autoblocks, or just ignore any number not on their contacts list and area code. If someone does answer and gets a "hi, this is not a sales call", the "end" button on the phone gets pressed by instinct, just like one's hand draws back if they touch a hot pan.
".. the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven't figured out how to replace it..."
Here's a crazy idea: let's have everyone vote, and then see what the results are before we report on it?
Or even weirder: instead of micromanaging a candidate's positions based on what they think the public wants to hear, have the candidate state what they actually think, and let the public judge them (shock!) on their actual beliefs? Do they even remember what they think themselves still?
I know, I'm so old-fashioned.
-Styopa
The summary tries to blame this on the FCC for "interpreting" the Telephone Consumer Protection Act (TCPA) to apply to survey calls to cell phones. The law itself (47 USC 227), not some rogue FCC interpretation, says no auto-dialed or prerecorded calls to cell phones without express consent. Period. No exceptions for "surveys." No exceptions for "get out the vote" political calls either.
Considering that the major change in campaigning strategy over the last 15 years has centered around using statistical techniques to hack an election, this probably is not a bad thing at all. It means that defining a wedge issue and engineering the entire political discourse toward that wedge might have some uncertainty. Maybe the candidates can talk about things that they actually believe.
first you must discredit the polls.
No reliable polling data? The horrors!
Instead of focusing on the horse race (who's ahead? who's falling behind?), do you think the media will discuss what candidates actually say and do, maybe even compare and contrast their stump speeches with their actual record and/or accomplishments?
That would truly be "we inform, you decide."
Lose = not win
The opinion polling industry here in the UK got got last month's General Election badly wrong. Not only did almost all of the pre-election polls (conducted by a wide range of companies, some of whom use telephone surveys while others use an online approach) get the vote-distribution wrong, over-estimating Labour support and under-estimating Conservative support, but they also misread the mapping of those vote-shares into Parliamentary seats (which is, admittedly, not always simple in the UK's first-past-the-post electoral system).
Only the exit-poll conducted on the day of the election itself got relatively close to the actual result (and even that under-estimated the scale of the eventual Conservative victory).
There's a major industry post-mortem in progress at the moment, which is scrutinising various aspects of previous methodological orthodoxy. UKpollingreport has a fairly good write-up of the state of play here.
There's been a fair degree of political acrimony about the inaccuracy of the pre-election polling. In particular, there have been questions raised about whether inaccurate polling caused the parties or the voters to change their behaviours in a way that accurate polling (or no polling) wouldn't have. There are also some calls for the UK to follow the example of some continental European countries and ban the publication of opinion polls in the 2-3 week period before an election.
One other point worth noting is that there was one particular data-analytics organisation (sorry, can't find the link right now) which looked at the raw data from the opinion polls and made a call a few days before the election which predicted the outcome fairly accurately.
Nate Silver called it badly wrong, in this instance.
The Demise of polling could be the best thing that ever happened to US Politics.
It would remove an essential tool from the typical two faced campaign tool chests. You can't just say what people want to hear when you have no idea what that is.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
In British Columbia, the media is not allowed to report poll results within 30 days prior to an election. Politicians can have a poll done, but they can't reveal the findings. I'm sure that that two-edged sword, the U.S. Bill of Rights, would never permit such a "free-speech" restriction in the U.S.
As far as I'm concerned, polling is a tool used to sway voters and manipulate voter turnout. Imagine my disgust way back in 1980 when driving to the voting 1/2 hour before opening time to hear over the radio that NBC had declared "Raygun" the next president of the U.S. Many of my (then) young friends told me that they hadn't even bothered voting because they didn't think that their vote would count given the polling numbers that were flooding the media.
I've been burned by interminable pools too. I'd be a lot more willing to answer polls if the people on the phone started with something like, "Hello. We're doing a political poll that has X questions and will take about Y minutes. Are you interested?" Y would be 3-4 minutes tops. I'd answer that type of poll.
1) your friends are idiots if they let their intentions change due to what some poll says.
2) This ain't a new phenomenon, at all
3) A poll result does not necessarily mean that it matches the election result. See also "Dewey Defeats Truman"
3) Reagan won by a frickin' landslide in both elections, so it's not as if the media outlet had jumped any guns.
Quo usque tandem abutere, Nimbus, patientia nostra?
Just imagine a world where instead of tailoring their message to what the people say they want to hear, they have to put out a message of what they really plan on doing and the people make their voting decisions based on that. We also need a much cheaper and easier method of recalling elected officials. Right now they really couldn't care less about offending the voters because they have a guaranteed job for the next several years, and by the time the next election rolls around most folks have forgotten what wrongs they've done. If a supermarket manager did something on the same relative scale of wrongness that some of these congresscritters do weekly, they'd be out of a job before the sun set. Congress needs to have the same immediate fear for their jobs. After all, can't kill them, can't staple bologna to their foreheads.
Political polls are exempt from do-not-call lists and every Sept-early November my phone rings several times each evening. I can ask to not be called and I will continue to get called over and over.
This year I've decided if I can't make phone polls stop calling I will actively work against them by lying my ass off. I'll tell them I'm voting for Darth Vader because he's honest about where he stands on social issues and foreign policy. The most important issue in this election season is freeing minds from the Matrix.
It also never hurts to answer every question with "Hodor"