Research shows what it doesn't show

If you don’t have a phone, you can’t hang up on them.

Landlines have become quaint. It is almost funny to think that, here we are in 2010 and yet, some people still use those old style telephones that require wires to plug into the wall in order to work. Way back in 2004, more than 90% of households had a landline. In 2009, 25% of households were cell phone only.

Why is this a political story,” you might ask? It is against federal law* to use the automated dialers, which most pollsters use, to call cell phones. Add to that the number of households who primarily use their cell phones (cell dependent) and screen their landlines, and you begin to see a larger problem accurately projecting poll numbers.

What’s the big deal, they can still reach plenty of people with landlines,” you might ask? Sure, except the just released and updated Pew Research Center study also shows that people in cell only and cell dependent households are different than landline households – just calling more landlines won’t get the right answers.

The differences are startling. Cell only households are much younger: only 7% of younger households (under 30) have a landline and only 26% of households, ages 30-49 have a landline. They are 11% less likely to have a college degree; make less money (they are younger, duh?); half as likely to be married; three times more likely Hispanic; two-times more likely African-American; and more than twice as likely male. They also are more likely to to support Democratic candidates.

Pollsters are pretty smart and factor this in when they call, right,” you might ask? They are smart, but most surveys are conducted with automated dialing only calling landlines and the non-coverage bias is not factored. Some surveys make an attempt to compensate by dialing a sample of expensive-to-manually-call, cell phone lists (in pollster lingo, they are called, “full dual frame surveys”). According to the Pew findings even the full dual frame surveys have a real problem.

“Because the decline of landline coverage has not been uniform across demographic groups, some key subgroups in surveys based only on landlines may be severely underrepresented, making reliable estimates of attitudes or behaviors among those groups difficult or impossible to obtain. For example, respondents ages 18-29 now constitute just 7% of a typical landline sample, less than one-third of their proper proportion in the population according to the latest American Community Survey estimates (22%). The shortfall is not limited just to the very young, in part because many people maintain phone status as they age, and in part because even older adults are abandoning landline service. Consequently, the percentage of adults in their 30s and 40s represented in landline surveys now falls 12 percentage points short of the parameters (26% vs. 38%). As a result, adults 50 and older are significantly overrepresented in landline samples, comprising 66% of the average landline sample when they should be only 40% of the sample.

”The coverage issue also affects other demographic variables in addition to age. Compared with dual frame samples, landline samples yield relatively fewer cases among Hispanics, an important and growing portion of the U.S. population. Renters also are more likely to be missed by landline surveys.

Be warned next time you hear the results of some new survey, it will be wrong.

For a full copy of the Pew Research report, click here.


*Telephone Consumer Protection Act (TCPA). Survey researches are, however, allowed to call the 91+ million households who have signed up to be on the do not call lists.

5 thoughts on “Research shows what it doesn't show

  1. Gita

    I had direct experience with this phenomenon four years ago. I was on the executive committee in my county for the political party of my choice; the state headquarters handed me a list of registered voters in my county so that I could begin calling to rally the troops and get out the vote.
    The list was only two years old.
    But one in four land line phone numbers were no longer in service, and in some precincts it was one in three. Even when we were able to get cell numbers, from relatives or neighbors, guess what? Between August (when I started working the list) and November, more than half the people had changed their cell phone providers and numbers. I joked at the time that we need to put GPS tracking chips in people’s molars to find them. Then I realized, that probably wasn’t funny.

  2. Monica Smith

    I’ve been arguing for at least four years that the political polling should be ignored, but the lack of accuracy is the least of my reasons. Republicans have been using polling “results” as a supposedly “objective” measure of public interests and attitudes for decades, in order to shape the attitudes of their followers and depress the electoral participation of the opposition — i.e. people who don’t agree with them and don’t buy into the propaganda. A lot of effort has gone into negating universal suffrage by discouraging interest and voter participation. Even the objective behind publicizing the scandalous behavior of politicians is not to sanction such behavior, but to turn the electorate off politics. Dirty tricks on election day are much too late to be really effective. All they do is give the temporarily attentive press something to report.

    Also, prediction that turn out to have been wrong garner more attention than those that were right. So, there’s another example where failure is more profitable than accuracy. When profit becomes the objective, then profit can no longer be a measure of value and quality.

    1. Lee Leslie Post author

      Right on, Monica. Here’s a quick for instance. A major independent political poll is being conducted over the next week or so in Georgia. Coincidentally, Roy Barnes just started a major 2-week ad blitz. The primary is not until July. Think this about Thurbert Baker, DuBose Porter, Carl Camon, David Poythress and creating an enough bias to squeeze their contributors hoping some will drop out? Or to put the fear in Johnny John Oxendine and company?

      More cynically, right wing pollsters know that a landline poll will over estimate their candidates and issues by 3-7%. Generally, enough to swing the result and be quoted for a news cycle or two.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.