Slashdot Mirror
Will Cellular Phones Skew Survey Results?
FriedEgg writes "Recently, many people have started to forgo traditional landline telephones in favor of cellular phones only. This presents a problem for telephone pollsters who are prohibited by the FCC from calling cell phones with automated equipment, and from calling people for whom receiving the call costs money. While they recognize the exclusion of cellular only users can skew their results, they're not yet sure how much... because they're unable to survey cellular only users to find out their demographic information. Some evidence does indicate the frequency of cellular-only is highest among 18-24 year olds, traditionally the hardest to survey anyway. If the problem grows worse, it's possible we could end up with a "Dewey Defeats Truman"-like situation where the telephone poll results were skewed because Truman supporters were less likely to own a telephone."
A lot of times people will or will not vote for someone based on how well they are doing. People like winners.
Another example would be the 2000 election. Bush would have won CA, since a lot of CA liberals (our favorite appelation for them) were going to vote Green since CA is sufficiently liberal. When they discovered what was happening, many switched to Gore.
Also, money, which like it or not is invaluable for elections and free speech (thank you for abridging free speech through campaign finance reform), is given to people leading in polls. Leading in the polls makes it easier to raise money.
So inaccurate polls can really make things screwy.
I would have to say that there is already an effect from the number of people with Caller ID or who screen their calls with answering machines. I know I don't bother picking up if there's no caller info or it says "GALLUP POLL".
The question for the market research is:
Does the universe of people who have cell phones correlate to the universe of respondents that I'm trying to reach or does it introduce bias? (I know just enough to get myself in trouble here...)
e.g. if 55% of your population is A, and 45% is B, and the cellphone population is also 55% A / 45% B, then it will have no impact on the results. OTOH, if the cellphone population is 75% A / 25% B, then there is going to be bias that will have to be corrected for in order to extrapolate back to the main population. (Guaranteed there is some wrong terminology there...)
Wolde you bothe eate your cake, and have your cake?
Some excerpts from the first article:
In an unscientific poll last month by CNN and Money magazine, 41 percent of more than 11,000 voluntary participants said they would be inclined to pull the plug on their land lines and go entirely wireless.
Unfortunately no reason is stated, but I have a feeling that "so these stupid pollsters don't call and pester me" is one of the leading reasons.
A recent Sprint Wireless survey of more than 500 college students found that half are cell-only customers.
No kidding, Sherlock. Think about it. Students are always on the move, changing addresses frequently. It is MUCH easier to have a cell phone only than to deal with changing land lines every 6 months.Students are also the demographic group with too many other things to do and think about than answer some stupid survey.
"That makes it an opt-in situation where the cellphone user somehow signs up to allow researchers to call," she said. "Who's going to do that?"
Like....DUUUUUUH! Get a clue already! YOU'RE UNWANTED!
People say I'm crazy, I got diamonds on the soles of my shoes...
karma, karma, karma, karma, karma chameleon....
The problem is that many of the people in this business aren't mathematician and if they are then they are at best only second rates ones.
This poblem doesn't appear only in public survey - in fact most "statistics" in natural and political sciences are usually fucked. Most mathematicians don't care - we know that significance tests don't really give any viable results etc. So if people aren't interested in maths or aren't willing to invest time to understand it, then let them believe their rubbish.
Nevertheless all this whining about "skewed" surveys is meaningless, it's like complaing that the moon is made from bad tasting cheese.
Owner of a Mensa membership card.
Why can't the wireless companies set up a way to allow the caller to pay the charges? Politicians would get more accurate surveys, wireless companies would get another revenue stream, polling companies would finally be able to reach those elusive cell-only customers... everyone wins!
Well okay, the consumer gets screwed with a bunch of survey calls that they don't want... but really, how important are the consumers anyway?
I am NOT a man!
I am a free number!
I'm somewhat reminded of the old Animaniacs skit from years ago.
Seriously, I get a lot of calls from telemarketers and poll people. They annoy me more than other people because I sleep during the day, but for the most part I accept it as a consequence of being in the phone book. Anyone can look up my name, phone number, and address in the local white pages, run to a terminal, and google one or all of those to bug me.
Every once in a while if the poll topic sounds interesting I'll do the poll-- well, OK, so I did it twice in the last six months, and really only because the interviewer sounded cute-- but the point is that I sometimes do the telephone polls in an attempt to influence the bias one way or another. If it's a poll I feel strongly about (like, say, the grocery stores in my area-- all of which are crap, thank you very much), I'll do it with the attempt to get something better in the area. If it's about gaming or electronics or somesuch, hell yeah I'll do it-- I like talking about games, and if you say that doing the survey gets me free stuff I'll ask to do it twice.
The point is that not everyone unconditionally says "fuck off" to anyone who doesn't get recognized on their Caller ID. So telephone polls are skewed, simply because the people who do them anyway are either too dumb to say no or think they can change the results with it. Which one of these two categories I fall into is left as an exercise for the reader.
"Why Subscribe?" Good question...
As I say, I can't really see a need for a landline.
Surprisingly, the parent didn't post any links to the famous picture of the debacle. Well, I guess it turns out there are a few of them.
Here are three, including a wide shot I hadn't seen before..
W
-------------------
This is my SIG. There are many like it, but this one is mine.
Who cares if pollsters can't get accurate results, although newspapers, TV seem to love polls I'm sure that most people couldn't care less what the polls say and are happy to wait until the results of ( whatever ) are announced offically.
I'd suspect that the fact that you're an American has something to do with why you were taught about it in your stats class.
I'm from the UK, and have had a fairly extensive mathematical training, including dedicated theoretical statistics classes. This was never even mentioned.
It's hardly surprising, though - there are plenty of ways of addressing the topic, why choose one poll from (then) 45 years ago that took place in another country, when there are other, more recent examples to use?
It's official. Most of you are morons.
In Denmark we got number portability for land-lines a couple of years ago, so area codes do not necessarily tell you where people live. Since telephone pollsters use area codes to make sure they have geographical coverage, this presents a similar problem.
As for the "Dewey Defeats Truman" episode, my understanding is that this was exit polling conducted by the Chicago Tribune in a massive, first of its kind, nationwide effort. Exit polling, as I'm sure you know, is generally highly accurate, since it reflects actual voters, not people who claim they're going to vote some months or weeks hence (or not).
My father worked at the Tribune in 1948 and described to me the massive bank of telephone operators they had there, with stringers throughout the nation phoning in exit poll results, which were then tabulated by electromechanical adding machines. It was a very labor intensive effort.
Anyway, despite its apparent "failure" -- i.e. in predicting the winner -- I believe that the Tribune's use of exit polls was actually looked at as quite a success, because it did produce good numbers. Their problem wasn't the data they received from exit polling, it was hubris: they looked at the data, which showed a very close election, and decided to predict the winner despite the closeness. They gambled, and "lost," but didn't lose much.
Patrick Doyle
I mod down every jackass who puts his moderation policy in his sig. Oh, wait a sec....
A survey held at my university on the day after student government elections showed that 30% of those surveyed had voted. The actual turnout was only 10%. Without the slightest embarassment, the poll takers reported a "margin of error" of 2% on their numbers.
There are three very different types of people in the world: 1) those that lie on surveys, 2) those that avoid surveys, and 3) those that actually participate and tell the truth on surveys. I would suggest that the errors and biases introduced by groups 1 & 2 make the data from group 3 all but meaningless.
Two wrongs don't make a right, but three lefts do.
I have a brother-in-law who sells vacuum cleaners. He told me that his company sets up sales appointments for him. I asked him how his company finds these people, and he responded that the company starts out with a "survey" call. If "they qualify," then they will get a second phone call asking if they would like a visit from a company representative to tell them more about these vacuum cleaners.
This company argues that those who participated in the initial survey have a "prior business relationship" with the company, and so even if they are on the National Do-not-call Registry, the company can still make a soliciting call to them. The survey is just a front to get around FCC regulations. Hence, I personally will never respond to a survey call, because I cannot trust the intent of the survey-takers.
On that note, I got a call on behalf of my local state trooper organization a couple of weeks ago. After some small talk, the guy on the phone asked, "We offer a $45, a $50, and a $60 contribution amount to the state trooper fund. Which of these amount would work best for you?" Well, first of all, I *hate* coercive questions. I told him, "I don't know; I'll have to think about it." He responded along the lines of, "Well, we do have a minimum contribution option of $15. Why don't I send you a letter about this, and you don't have to commit to anything at the moment." Okay, whatever. So I got a letter in the mail thanking me for committing to giving them money, and I noticed (in small print) a mention of thanks for this telemarketing company for helping the state troopers raise these funds.
That did it for me. Under no uncertain circumstances will I ever deliberately justify a telemarketing firm's existence. Part of my contribution would go to help support the telemarketers. I shredded that letter on the spot, and when that marketing firm calls again, I'm going to tell them in a very unpleasant way exactly why they never heard back from me (hint: it will probably emphasize exactly what I think about telemarketing types).
An unjust law is no law at all. - St. Augustine
I would think that the increased use of caller ID would already be making phone polls almost worthless. Who answers their phone anymore if they don't know the caller? There is a certain population that just doesn't want to participate in surveys any more and there is another group that won't be bothered with phone calls. So who is answering these survey phone calls? That's a piece of data I would like to see.
Of course, the news media outlets (radio, television, newspaper, and web) like the polls because they make for quick headlines and short articles. Thus, the polls results become a force driving opinion disproportionately.
Of course, I'm not a statistician, so I'm probably just blowing smoke out of my ass.
Yeah, I'm as old as my UID would suggest.