How to Read the Polls

Morning Jay: How to Read the Polls

Jay Cost

August 24, 2012 6:00 AM

In every presidential cycle, there is a debate about partisan identification in polling. Conservatives complain about too few Republicans being sampled; pollsters, journalists, and liberals respond by saying it is inappropriate to weigh polls by party identification.

What to make all of this?

The primary issue relates to the population that polls are sampling. There are several populations to choose from, and we can conceive of this as a series of concentric circles.

The idea here is that each circle represents a different population that can be sampled – but with a twist. The adult population includes all other sub-populations, the registered voter population is smaller than the adult population but includes likely and actual voters, and so on. And of course we cannot poll the actual voters until after they have voted!

Each population is worth polling, albeit for different questions. For instance, if you want a gauge of “consumer confidence,” you naturally sample the adult population. If you want a measure of how voters think about the state of the economy, a poll of adults would not be your best bet.

The reason that polls become a partisan football is that the further out on the circles you move, the more Democratic the poll usually gets. Likely voter polls usually have fewer Democrats than registered voter polls, which have fewer Democrats than adult polls.

Thus, the conservative case against “oversampling” Democrats is not that a pollster has included more Democrats than he should have in his sample, but that the population he is sampling has more Democrats than will come out to vote on Election Day.

This is how pollsters can be honest brokers, employing best practices, and still produce polls that overestimate the Democratic standing.

So what is the solution? At first blush one might conclude that the best approach is just to move inward through the circles – from registered voters to likely voters. Yet this approach does not necessarily produce a clearer picture of the ultimate electorate. We are still quite far from Election Day and a large portion of the voting public has yet to engage, meaning that those polls could still be more Democratic than the actual electorate (or even more Republican)

Absent some panacea, we have to approach the polls with prudence. Above all, we have to remember that the population we care most about – the actual voters – cannot be polled. So, all polls must be examined with caution.

We also have to keep history in mind. Over the last quarter century, party identification on Election Day has actually been quite stable in presidential elections.

On average, Democrats have enjoyed a 3-point advantage going back to 1984; in most elections, the edge falls between D+1 and D+5. Some argue that there is a recent trend that favors the Democrats, as 2008 saw the highest Democratic advantage since 1980. On the other hand, 2004 saw the best Republican year in generations.

A lot of the movement in party identification from cycle to cycle is not the two sides “turning out their bases,” as is commonly assumed. Instead, it has to do with how the marginal partisans on both sides are reacting to the national political climate. Remember: Something like 35 percent of the population calls itself “independent,” but only about 10 percent has no party affiliation whatsoever. That means there are a lot of "hidden" partisans who can "emerge" in good years for their sides. For instance, in a good year for the GOP, regular Republican voters who often think of themselves as independent will call themselves Republicans, and so the percentage of Republicans in the electorate will rise. In a bad year for the GOP, they’ll call themselves independents and the percentage of Republicans will fall. Same goes for the Democrats.

This actually explains much of the Democratic advantage 2008. A 20 percent increase in the black vote relative to 2004 helped boost the Democratic share of the electorate to 39 percent; meanwhile, some Republican voters started calling themselves “independents.” They still voted for McCain, however, which explains why Obama only won the independent vote by 8 points, compared to the 20-point victory House Democrats enjoyed in 2006.

Applying this historical perspective to today’s polls, I think it is unlikely that we are going to see a Democratic advantage equal to or larger than the D+7 Obama enjoyed in 2008. I see something closer to the historical average of D+3. That’s just a guess, albeit an educated one.

And when I look at partisan splits in the polls, the farther a poll moves away from D+3, the less I am willing to take a “naïve” view of it, i.e. just look at the head-to-head margin. Instead, I look closely at the way the independent vote is leaning, as that is a good gauge of where the race actually stands. As a general rule of thumb, if you see President Obama leading in the overall numbers but losing among independents, it stands to reason that the poll is more Democratic than what we shall see on Election Day.

Another strategy is to look at trends within polls. That can provide a sense of the motion within the race without getting hung up party identification.

Finally, an average of all the latest polls can ameliorate the problem, to some extent. Compare Obama’s average lead or deficit to the average Democratic identification edge (among the voters), and you can get a solid gauge of the state of the race. For instance, consider these polls currently included in the RealClearPolitics average. All of them provide the head-to-head numbers between Obama and Romney as well as the partisan identification of those included in the poll.

Right now, the average across the polls is about a one-point lead for Obama; there's also a Democratic registration advantage that fits in with the long-term trend. That suggests to me that right now, this is a pretty solid take on the state of the race.

As some polls cycle in and other polls cycle out of the RCP average, that might change. A few weeks ago, Obama had a larger lead but the Democratic identification advantage was also larger on average. In that case, the best bet was to discount his margin over Romney by a little bit.

The best advice is just to remember that it’s still only August. We are some 75 days from the election and that means it is very difficult to get a read on that all-important inner circle, the elusive subsample of the adult population that actually will vote. So, all polls need to be taken with a grain of salt. These strategies are just meant as practical ways to deal with what remains a basically intractable problem.

Jay Cost is a staff writer for THE WEEKLY STANDARD and the author of Spoiled Rotten: How the Politics of Patronage Corrupted the Once Noble Democratic Party and Now Threatens the American Republic, available now wherever books are sold.