Puzzled by a Plethora Of Polls? Remember: Follow the Trends
“Voters Shift in Favor of Kerry” (Los Angeles Times, June 5-8). “Bush Edges Kerry” (Fox News, June 22-23). “Bush’s Rating Falls to its Lowest Point” (CBS/New York Times, June 23-27). “Bush Makes Gains on Personal Traits, Some Issues” (Annenberg Poll, June 8-21). Bush “Down but Not Out” (NBC/Wall Street Journal, 6/28).
[IMGCAP(1)]Confused? It’s not surprising. Political polls over the past month have been more contradictory and difficult to understand than the voting record of Sen. John Kerry (D-Mass.). Some partisans on both sides, unhappy with a particular poll, will see bias behind differences — particularly widely varying differences — in poll numbers like these.
Certainly, there have been instances when bias has influenced the outcome of a poll, but usually disparities in poll results can be attributed to more mundane reasons. First, methodologies differ between pollsters. From my experience, some are more defensible than others. It is important when looking at surveys to know the party-identification and ideological makeup of the survey sample.
Polls that fail to reflect the election-day electorate run a higher risk of creating a false picture of the political state of play. Unfortunately, when they are media polls, they are often accompanied with an overstated headline or broadcast copy that only adds to a poll’s credibility problems.
Second, poll results can be affected by timing. Some polls are taken over the course of two to three weeks; others over three days while tracking is done every night. Comparing polls without regard to how and when they were taken is risky business.
Finally, one of the most important factors in the outcome of political polls are the questions themselves. Just as polling editors’ methodologies differ, so does the content and design of their questions.
Here again, bias can creep into the process, and sometimes does. But pollsters can legitimately use different approaches to try to answer the same questions. The key is to not compare apples and oranges — that is, to avoid drawing conclusions based on data drawn from different polls using different questions on similar issues.
So, what’s a political poll watcher to do? Stop bouncing from poll to poll and start looking at the trends. In this presidential election contest, there is no better indicator of voter trends than the right-track/wrong-track question because, unlike other recent presidential elections, there are a number of major issues that will impact the context of this year’s election.
In 1996 and 2000, there were no real external issues that were dynamic enough to affect the context of the races. This election will take place with big issues in play: the state of the economy; the war against terror, especially in Iraq; and how people respond to the new prescription drugs program.
While pollsters will test the strengths and weaknesses of Bush and Kerry on each of these issues through issue-handling questions, the right-track/wrong-track question, along with overall presidential job approval, measures this broader issue environment and is a key indicator of Bush’s position.
With job creation having been a lagging political indicator and with events in Iraq hotter than expected, the right-track/wrong-track numbers in most polls this spring were worrisome for the Bush campaign. Many Democrats have been boasting that the right-track/wrong-track numbers all but assure Kerry’s election.
But they’ve been doing too much navel-gazing and not enough number-crunching. Looking at one poll or another doesn’t tell the real story. For an accurate picture, you have to look at trends — both across the polling universe and within individual polls. And there, the trends for Bush are solidly up.
Here are just a few examples of the right-track/wrong-track numbers in several key polls. The plus figure refers to the percentage-point change in wrong track minus right track from one poll to the other:
Annenberg + 15
June 8-21 40-50
May 17-31 33-58
CBS/New York Times + 14
June 23-27 36-57
May 20-23 30-65
NBC/Wall Street Journal + 5
June 25-28 36-48
May 1-3 33-50
Democracy Corps +5
June 14-17 41-55
May 10-13 37-56
Associated Press /Ipsos +4
June 7-9 40-56
May 3-5 38-58
Gallup +6 (slightly different wording: Satisfied-dissatisfied)
May 2-4 36-62
June 3-6 39-59
While the numbers vary from plus-4 to plus-15, the trend lines within each of these polls are positive news for the Bush campaign. So are the ballot-test numbers, which have been consistently competitive.
Not all trends have been positive for Bush. Clearly, the diminishing support for the war in Iraq is cause for concern, but it’s important to remember that most polls have not yet measured the impact of the handover of sovereignty in Iraq, the arraignment of Hussein or, for that matter, the latest positive numbers on jobs and the economy. When they do, however, those poll results should be assessed in the context of previous polls — keeping the methodology, timing and actual questions in mind.
It’s going to be a long, hot summer with hundreds of polls. Don’t make yourself crazy. Follow the trends, not the headlines.
David Winston is president of The Winston Group, a Republican polling firm.