Slashdot Mirror
Political Polls Become Less Reliable As We Head Into 2016 Presidential Election
HughPickens.com writes: Cliff Zukin writes in the NY Times that those paying close attention to the 2016 election should exercise caution as they read the polls — election polling is in near crisis as statisticians say polls are becoming less reliable. According to Zukin, two trends are driving the increasing unreliability of election and other polling in the United States: the growth of cellphones and the decline in people willing to answer surveys. Coupled, they have made high-quality research much more expensive to do, so there is less of it. This has opened the door for less scientifically-based, less well-tested techniques.
To top it off, a perennial election polling problem, how to identify "likely voters," has become even thornier. Today, a majority of people are difficult or impossible to reach on landline phones. One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone. To complete a 1,000-person survey, it's not unusual to have to dial more than 20,000 random numbers, most of which do not go to actual working telephone numbers.
The second unsettling trend is rapidly declining response rates, reaching levels once considered unimaginable. In the late 1970s, pollsters considered an 80 percent response rate acceptable, but by 2014 the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven't figured out how to replace it," concludes Zukin. "In short, polls and pollsters are going to be less reliable. We may not even know when we're off base. What this means for 2016 is anybody's guess."
To top it off, a perennial election polling problem, how to identify "likely voters," has become even thornier. Today, a majority of people are difficult or impossible to reach on landline phones. One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone. To complete a 1,000-person survey, it's not unusual to have to dial more than 20,000 random numbers, most of which do not go to actual working telephone numbers.
The second unsettling trend is rapidly declining response rates, reaching levels once considered unimaginable. In the late 1970s, pollsters considered an 80 percent response rate acceptable, but by 2014 the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven't figured out how to replace it," concludes Zukin. "In short, polls and pollsters are going to be less reliable. We may not even know when we're off base. What this means for 2016 is anybody's guess."
Who wants to be Nate Silver will be able to make sense of the polls?
Still some interesting points, and yes we may reach a point where polls actually have no predictive value. But I doubt we've gone from "100% accurate if you know how to interpret them" to 0% in 4 years ;-)
What do you expect. Figures don't lie, but liars can figure.
INSERT INTO comment VALUE('Doh!') WHERE user='you';
One problem is that the 1991 Telephone Consumer Protection Act has been interpreted by the Federal Communications Commission to prohibit the calling of cellphones through automatic dialers, in which calls are passed to live interviewers only after a person picks up the phone.
I know some people that the pollers could outsource to that have seemed to have found a very easy workaround to this problem.
"Hi, this is Rachel from polling services....."
The only thing necessary for evil to triumph is for it to be pitted against a slightly greater evil
... it means you wait until the votes are counted to declare a winner instead of when the press tells you who the winner is.
Post Citizens United we're going to get the best government that money can buy.
One reason why polling companies can't get usable info is that end users tend to be constantly barraged by robocalls, be it the GE security system, "polls" which actually turn out to be scammy sales pitches, or many other types of scams.
Because of this, people either use apps like Mr. Number which autoblocks, or just ignore any number not on their contacts list and area code. If someone does answer and gets a "hi, this is not a sales call", the "end" button on the phone gets pressed by instinct, just like one's hand draws back if they touch a hot pan.
".. the response rate has fallen to 8 percent. "Our old paradigm has broken down, and we haven't figured out how to replace it..."
Here's a crazy idea: let's have everyone vote, and then see what the results are before we report on it?
Or even weirder: instead of micromanaging a candidate's positions based on what they think the public wants to hear, have the candidate state what they actually think, and let the public judge them (shock!) on their actual beliefs? Do they even remember what they think themselves still?
I know, I'm so old-fashioned.
-Styopa
Perhaps ask people go to a place and let their preference be known. Let's call the place a "polling place" and let's call their preference, say, a "vote". We can get rid of the term "poll" and use some new fancy term like "election".
But, polling really does need to change with people's communication preferences. ID verification was ALWAYS a problem on phones. I think that knocking on doors, trusted e-mail, text messages, and other alternatives exist. Harder to do, but oh well. If you want good data, its ALWAYS really hard to do. Good data is very difficult to come across. All data is wrong, but sometimes it tells you something interesting... (something like that...)
The talking heads and candidates care who is "leading in the polls". I don't. I choose my candidate based on what is best for ME and then I ALWAYS vote. I ALWAYS lie to pollsters. Or do I?
"If you want to improve, be content to be thought foolish and stupid." - Epictetus
The summary tries to blame this on the FCC for "interpreting" the Telephone Consumer Protection Act (TCPA) to apply to survey calls to cell phones. The law itself (47 USC 227), not some rogue FCC interpretation, says no auto-dialed or prerecorded calls to cell phones without express consent. Period. No exceptions for "surveys." No exceptions for "get out the vote" political calls either.
Considering that the major change in campaigning strategy over the last 15 years has centered around using statistical techniques to hack an election, this probably is not a bad thing at all. It means that defining a wedge issue and engineering the entire political discourse toward that wedge might have some uncertainty. Maybe the candidates can talk about things that they actually believe.
first you must discredit the polls.
I have to wonder how much of this has to do with 'push polling'. I have a zero tolerance for this practice and I will shut down a survey call in a heartbeat if it appears to be headed in that direction. I have probably ended some legitimate calls because of this.
Add to that the following:
I certainly don't have to wonder why people are so skeptical of surveys.
No reliable polling data? The horrors!
Instead of focusing on the horse race (who's ahead? who's falling behind?), do you think the media will discuss what candidates actually say and do, maybe even compare and contrast their stump speeches with their actual record and/or accomplishments?
That would truly be "we inform, you decide."
Lose = not win
The opinion polling industry here in the UK got got last month's General Election badly wrong. Not only did almost all of the pre-election polls (conducted by a wide range of companies, some of whom use telephone surveys while others use an online approach) get the vote-distribution wrong, over-estimating Labour support and under-estimating Conservative support, but they also misread the mapping of those vote-shares into Parliamentary seats (which is, admittedly, not always simple in the UK's first-past-the-post electoral system).
Only the exit-poll conducted on the day of the election itself got relatively close to the actual result (and even that under-estimated the scale of the eventual Conservative victory).
There's a major industry post-mortem in progress at the moment, which is scrutinising various aspects of previous methodological orthodoxy. UKpollingreport has a fairly good write-up of the state of play here.
There's been a fair degree of political acrimony about the inaccuracy of the pre-election polling. In particular, there have been questions raised about whether inaccurate polling caused the parties or the voters to change their behaviours in a way that accurate polling (or no polling) wouldn't have. There are also some calls for the UK to follow the example of some continental European countries and ban the publication of opinion polls in the 2-3 week period before an election.
One other point worth noting is that there was one particular data-analytics organisation (sorry, can't find the link right now) which looked at the raw data from the opinion polls and made a call a few days before the election which predicted the outcome fairly accurately.
Nate Silver called it badly wrong, in this instance.
...where it's not really a poll at all, but a disguised negative campaign ad. At this point, if I don't recognize the number, I don't answer it. Period.
PHEM - party like it's 1997-2003!
Let's say polls were way off resulting in newspapers with headline errors. But the printed newspaper has gone wayside along with all those "hard to reach" people on landlines. However there is the internet. I did a screen grab of news website a week or two after the 2012 election that has a Romney infotainment article on the side, "we're confident we will win this election."
mfwright@batnet.com
The Demise of polling could be the best thing that ever happened to US Politics.
It would remove an essential tool from the typical two faced campaign tool chests. You can't just say what people want to hear when you have no idea what that is.
When Fascism comes to America, it will call itself Anti-Fascism, and tell you to give up your guns.
I agree that there could be positive effects if polling became useless, but there could also be negative effects. It's not clear to me which effects will be stronger. My guess is that it'll be a wash.
Two possible examples:
-I think that politicians have a good idea of what their base wants even without detailed polls. If politicians have no idea what the rest of the electorate wants, maybe politicians will pander even more to their base because it's a known quantity.
-When there's real hysteria, politicians don't need polls. Take the recent ebola scare/hysteria. I don't think politicians jumped on it because of polls.
In British Columbia, the media is not allowed to report poll results within 30 days prior to an election. Politicians can have a poll done, but they can't reveal the findings. I'm sure that that two-edged sword, the U.S. Bill of Rights, would never permit such a "free-speech" restriction in the U.S.
As far as I'm concerned, polling is a tool used to sway voters and manipulate voter turnout. Imagine my disgust way back in 1980 when driving to the voting 1/2 hour before opening time to hear over the radio that NBC had declared "Raygun" the next president of the U.S. Many of my (then) young friends told me that they hadn't even bothered voting because they didn't think that their vote would count given the polling numbers that were flooding the media.
I've been burned by interminable pools too. I'd be a lot more willing to answer polls if the people on the phone started with something like, "Hello. We're doing a political poll that has X questions and will take about Y minutes. Are you interested?" Y would be 3-4 minutes tops. I'd answer that type of poll.
1) your friends are idiots if they let their intentions change due to what some poll says.
2) This ain't a new phenomenon, at all
3) A poll result does not necessarily mean that it matches the election result. See also "Dewey Defeats Truman"
3) Reagan won by a frickin' landslide in both elections, so it's not as if the media outlet had jumped any guns.
Quo usque tandem abutere, Nimbus, patientia nostra?
Just imagine a world where instead of tailoring their message to what the people say they want to hear, they have to put out a message of what they really plan on doing and the people make their voting decisions based on that. We also need a much cheaper and easier method of recalling elected officials. Right now they really couldn't care less about offending the voters because they have a guaranteed job for the next several years, and by the time the next election rolls around most folks have forgotten what wrongs they've done. If a supermarket manager did something on the same relative scale of wrongness that some of these congresscritters do weekly, they'd be out of a job before the sun set. Congress needs to have the same immediate fear for their jobs. After all, can't kill them, can't staple bologna to their foreheads.
Political polls are exempt from do-not-call lists and every Sept-early November my phone rings several times each evening. I can ask to not be called and I will continue to get called over and over.
This year I've decided if I can't make phone polls stop calling I will actively work against them by lying my ass off. I'll tell them I'm voting for Darth Vader because he's honest about where he stands on social issues and foreign policy. The most important issue in this election season is freeing minds from the Matrix.
It also never hurts to answer every question with "Hodor"
I hate push polling. "what would you feel once you learned that Obama ate babies to keep his hair color?" "what would you feel once you found out that Bush is actually a lizard person".
the lower the response rate the more you can skew the results with bogus answers. Rather than hang up embrace the opportunity to shape the future positions of our government by creatively staking out you position. Don't think of it as a nuisance to be avoid but rather a chance to screw with politicians.
I'm a consultant - I convert gibberish into cash-flow.
What do they (pollsters) expect?
I got a call last Saturday morning from an "unknown caller" at 8:30am (which woke me up). I ignored it. Again at 9:30, again at 10:30. Finally I was near enough the phone (actually, Google Voice on an iPad Mini) to pick up. I asked who it was and got a personal name, then I asked who they were calling from and then they admitted it was "ANZ Research" or something that sounds like that. They said they were calling to get opinions on various political topics.
There's no way in hell I'm going to give survey answers to someone who's dumb enough to call before noon on the weekend. Google Voice lets me block numbers, which I suspect is why they disabled Caller ID, so they could sneak through. I refused to even confirm my name, and told them to take my number off their list and never call again.
I figured it was probably a push poll anyway.
The right to protest the State is more sacred than the State.
Just about everyone in the polling industry was significantly off-base in the recent UK elections. Literally no-one in the mainstream was calling the actual result in the run up to election day, as far as I know. The debate was all about who would be leading a coalition and how the electoral math would stack up to determine which parties would be likely to join. Even the party leaders changed their tune in the last days of the campaign to reflect an assumption that they wouldn't be governing alone and who they governed with would be a significant question.
Ironically, having won an unexpected absolute majority this may have left David Cameron and the Conservative Party leadership in a bit of a bind. I suspect some of the policies they were promoting before the election were things they didn't really want to do but advocated for popularity reasons, hoping that after the election they would be able to "reluctantly" negotiate away some of those commitments as part of a coalition agreement. Similarly, some of their more unpopular policies now won't have a partner party or two to act as scapegoats next time if those policies don't work out well. Given that their working majority is also very small, which leaves the leadership very vulnerable to disruption by rebel MPs on controversial issues such as Europe, ironically they might have been better off leading a strong coalition than winning. The pollsters and commentators and political journalists didn't consider any of these issues in much detail in their pre-election coverage, if they even acknowledged the possibilities at all.
If you disagree, post your argument. (-1, Overrated) isn't your personal censorship tool for views you don't like.
This decline in polls is VERY welcome. In fact, it's about the only way democracy will have a chance going forward.
The old style poll was democratic in nature: Do you believe in god? What car do you like? Why do you buy that brand of TV?
This was used without excess interpretation, and made available. Pollsters were sure to get a decent response rate, they were sure to get data that was statistically relevant, etc.
This gave polls a magic power: people believed they were true.
Where power goes, corruption follows.
Modern polls:
1)- Hardly ever list their rate of response.
This little trick allows the pollster to get what he wants his poll to say. It also can make for wildly sensational polls in general.
2)- Often is a form of advertisement or political mindfuck.
Ex: Are you Christian? Who will you be voting for in the next election?
Asking the questions in this order makes you more likely to vote for a candidate you perceive as more in line with the FIRST question. So if the first question is about God, you will be (statistically) more likely to actually vote, and more likely to vote for a Republican. No, no, you say, Reasons. First, you are quite possibly incorrect. Second, even if you are correct in saying that this can't possibly effect YOU, just pretend that it DOES effect everyone else, including those you know and love. They wouldn't do it if it did not literally make votes out of nothing.
3)- Way too meta, fuck that noise.
Current polls are often done with a bunch of other questions whose actual goals are to assess the level of corporate threat from different demographics and locations. You could think you are answering questions about kitchen cleaning products, but in reality many of the questions are just there as smoke screen (and no one cares about the thought or time you spend on them), and the "real" questions are to determine the level of political savy of a certain area, the likelihood of a future lawsuit, etc.
4)- Clearly not a civil service.
Polls used to be perceived something like a civil service- the companies, who have a responsibility to make the world a better place (this was not so long ago a thing- before the court decision saying that corporations had to act to maximize profits for shareholders), would get information on how to trade off reliability, quality, and cost, to make your life better. The politicians, always interested in Democracy, would figure out how to better represent people. The scientists, always interested in metrics, would figure out what you wanted and research in that direction. I don't know how true this actually was, but that was the PERCEPTION. Even if you don't keep up with all the psych tricks that any profitable or powerfocussed entity is employing, everyone sort of knows that no one is taking their opinions and making a better world for everyone- they are figuring out where you aren't looking so they can slash the pound of flesh with less of a fight.
This used to be a census. Now, it's intel.
Pollsters can fuck right off. With some exceptions- actual science still needs polls, and that's sad for them, but they are a rounding error in the giant race to "solve the democracy problem" that companies face (they don't like you voting elsewhere with your dollars) and that politicians face (they don't like that elections are not safe and determined in their favor).
It is rational to avoid polls, unless you are in possession of expert knowledge of the poll taker's integrity- never the case.
Stats from the last congressional election:
o 14% approval rate -- that was a poll
o 94% re-election rate -- that was actual voters.
o In the same election, national turnout was 36.3%.
I think the advent of the net's new accessibility to information outside of the laundered and agitprop driven channels, the money-based reasoning of SCOTUS, the lobbyist factor, the obvious malfeasance of Fox news, MSNBC, the blatantly unconstitutional legislation coming out of congress... and so on... all combine to give a very large portion of the people who might otherwise vote a sense that the system is so massively corrupt that there just is no point to it.
When you ask them -- polling asks them -- they tell you that. That's why the 14% approval rate.
But the only people voting are the droolers who watch MSNBC and Fox. They're agenda- and plank-driven (abortion! guns! perverts! terrorists! taxes! etc.) and that's driving them to or from one party or the other. And *they* are controlling the narrative here; that's why the polls just aren't -- and won't be -- working in the current context.
It's just an idea. But the data is hard data. Something has to explain it. It's too skewed to be any kind of random happening.
I actually do vote, but I have to say, it's pretty damned fruitless. This is a red (very red) state, and so that's the way the pendulum swings here, regardless of how I vote. If I vote progressive on something, it's not going to happen. If I vote conservative on something, it would have happened any way. This is not encouraging.
The only thing less productive than voting for progressive ideas here is voting for a third party candidate. Neither one does any good at all in terms of biasing the political system, but at least the progressive vote isn't buried or simply not mentioned. Sneered at, I think might be the most accurate term around here, actually. But they at least talk about it.
I've fallen off your lawn, and I can't get up.