Will Cellular Phones Skew Survey Results?

FriedEgg writes "Recently, many people have started to forgo traditional landline telephones in favor of cellular phones only. This presents a problem for telephone pollsters who are prohibited by the FCC from calling cell phones with automated equipment, and from calling people for whom receiving the call costs money. While they recognize the exclusion of cellular only users can skew their results, they're not yet sure how much... because they're unable to survey cellular only users to find out their demographic information. Some evidence does indicate the frequency of cellular-only is highest among 18-24 year olds, traditionally the hardest to survey anyway. If the problem grows worse, it's possible we could end up with a "Dewey Defeats Truman"-like situation where the telephone poll results were skewed because Truman supporters were less likely to own a telephone."

Comments

For Those...

[PakProtector]

...Of you who slept through History class, 'Dewey Defeats Truman' refers to the 1948 presidential election in which it was projected a man named Dewey would defeat encumbant president Truman, due to skewed mail-in poll results.

Link is to Historybuff for more info. Dewey Defeats Truman

The results are bogus already ...

[Dark$ide]

Aren't the results of any phone survey skewed already. If someone calls me unsolicited I'll tell them to go away impolitely.

Haven't the Americans just got a 50 million list of folks who don't want to be called.

Do the folks who fit those two categories have a common demographic?

Caller ID and call screening already do that

[WuphonsReach]

I would have to say that there is already an effect from the number of people with Caller ID or who screen their calls with answering machines. I know I don't bother picking up if there's no caller info or it says "GALLUP POLL".

The question for the market research is:

Does the universe of people who have cell phones correlate to the universe of respondents that I'm trying to reach or does it introduce bias? (I know just enough to get myself in trouble here...)

e.g. if 55% of your population is A, and 45% is B, and the cellphone population is also 55% A / 45% B, then it will have no impact on the results. OTOH, if the cellphone population is 75% A / 25% B, then there is going to be bias that will have to be corrected for in order to extrapolate back to the main population. (Guaranteed there is some wrong terminology there...)

Won't matter much for elections

[Anonymous Coward]

The 18-24 demographic is the one least likely to vote anyway

is it *so* hard to take a hint?

[tuxette]

People don't want to be called and pestered by pollsters. The only people that don't seem to understand are the pollsters.

Some excerpts from the first article:

In an unscientific poll last month by CNN and Money magazine, 41 percent of more than 11,000 voluntary participants said they would be inclined to pull the plug on their land lines and go entirely wireless.

Unfortunately no reason is stated, but I have a feeling that "so these stupid pollsters don't call and pester me" is one of the leading reasons.

A recent Sprint Wireless survey of more than 500 college students found that half are cell-only customers.

No kidding, Sherlock. Think about it. Students are always on the move, changing addresses frequently. It is MUCH easier to have a cell phone only than to deal with changing land lines every 6 months.Students are also the demographic group with too many other things to do and think about than answer some stupid survey.

"That makes it an opt-in situation where the cellphone user somehow signs up to allow researchers to call," she said. "Who's going to do that?"

Like....DUUUUUUH! Get a clue already! YOU'RE UNWANTED!

Does it matter ?

[Krapangor]

From a mathematical point of view most surveys are fucked anyway. Biased data collection, misunderstood mathematical rules and "correction formulas" if the results don't seem to fit.
The problem is that many of the people in this business aren't mathematician and if they are then they are at best only second rates ones.
This poblem doesn't appear only in public survey - in fact most "statistics" in natural and political sciences are usually fucked. Most mathematicians don't care - we know that significance tests don't really give any viable results etc. So if people aren't interested in maths or aren't willing to invest time to understand it, then let them believe their rubbish.

Nevertheless all this whining about "skewed" surveys is meaningless, it's like complaing that the moon is made from bad tasting cheese.

The obvious answer

[anon*127.0.0.1]

Why can't the wireless companies set up a way to allow the caller to pay the charges? Politicians would get more accurate surveys, wireless companies would get another revenue stream, polling companies would finally be able to reach those elusive cell-only customers... everyone wins!

Well okay, the consumer gets screwed with a bunch of survey calls that they don't want... but really, how important are the consumers anyway?

Poster gets it semi-right, Film @ 11

[Fortunato_NC]

The actual Dewey Defeats Truman headline was based more on early election returns than skewed polling. The famous error made concerning polling in the 1948 election was that Gallup simply stopped doing polls two weeks before the election and proclaimed that Dewey would win, not taking into account the massive sea change in public opinion that can occur over a two week span.

The famous example of a poll gone bad because of telephone ownership statistics was the poll used by the magazine The Literary Digest to predict the 1936 presidential election between FDR and Al Landon. The magazine mailed ballots to 10 million individuals with listed phone numbers and/or car registrations, and tallied up roughly 2 million returned ballots. Based on the results, the magazine predicted a Landon victory, while in fact the result was a 46-state landslide for FDR (and remember, there were only 48 states in 1936)!

The sample error in this survey was that telephone ownership and car ownership did not correlate to likeliness to vote in 1936. In 1936, at the height of the Great Depression, telephones and cars were luxury items that few people besides the wealthiest Americans could afford. The poor and lower class who were more likely to vote for FDR were not even sent ballots, so there was no way for their voices to be heard in this survey.

The impact of this flawed survey was such that within a short time after the publication of the survey, The Literary Digest would go out of business.

Re:Poster gets it semi-right, Film @ 11

[sidecut]

Thank god you corrected this! I just heard about this on Nova. That's a fascinating story.

As for the "Dewey Defeats Truman" episode, my understanding is that this was exit polling conducted by the Chicago Tribune in a massive, first of its kind, nationwide effort. Exit polling, as I'm sure you know, is generally highly accurate, since it reflects actual voters, not people who claim they're going to vote some months or weeks hence (or not).

My father worked at the Tribune in 1948 and described to me the massive bank of telephone operators they had there, with stringers throughout the nation phoning in exit poll results, which were then tabulated by electromechanical adding machines. It was a very labor intensive effort.

Anyway, despite its apparent "failure" -- i.e. in predicting the winner -- I believe that the Tribune's use of exit polls was actually looked at as quite a success, because it did produce good numbers. Their problem wasn't the data they received from exit polling, it was hubris: they looked at the data, which showed a very close election, and decided to predict the winner despite the closeness. They gambled, and "lost," but didn't lose much.

And some pictures...

[VValdo]

Surprisingly, the parent didn't post any links to the famous picture of the debacle. Well, I guess it turns out there are a few of them.

Here are three, including a wide shot I hadn't seen before..

W

The problem is big and real

[dybdahl]

The article doesn't provide many numbers, so I better provide some. In Denmark, there are more cellular subscribers now than fixed line subscribers. Of the 11 houses on our street, 2 only use cellular phones, and one of these has ADSL but no fixed line subscription.

In my nephiew's class, 2 families out of 24 don't have a phone number that belongs to a fixed line.

Currently, a fixed line costs 255 euros to create in Denmark, and 16 euros a month. It's cheaper per minute than a mobile phone, unless you call a mobile phone - in which case the mobile phone is cheaper per minute. Add to that the costs of buying a phone.

A mobile phone can be bought very cheap - used phones are normally 15-30 euros, there is no subscriptions fees and it's now down to 9 cents a minute outside workhours and down to 11 cents a minute in workhours. All prices include the Danish 25% Value Added Tax.

So in general, mobile phones are cheaper for most people, and more and more people are abandoning their fixed lines. The skew is already here and it's growing.