Like most of you, Im a huge consumer of polls. I rely on survey data, because they represent one of the few quantitative measures of candidate strength. Or do they?
For years, I have complained about national and local news anchors who dont seem to understand what polls mean and what a margin of error is. But increasingly, for me at least, its the proliferation of polls and polls of questionable value that are the problem.
Some observers assume the worst offenders are campaign pollsters, who presumably cook their numbers to show whatever they want to show. Not true. When Im presented with two polls, one conducted by a well-regarded Republican or Democratic pollster and the other by a nonpartisan pollster or state media outlet, I often place greater weight on the partisan pollsters numbers, especially if I regard the pollster highly.
Obviously, not all nonpartisan pollsters are equal (any more than all partisan pollsters are), and some of them regularly deliver what appear to be reasonable data. But Im increasingly suspicious of many polls.
Gallup is a first-class polling organization that has been around for years and seeks to maintain a certain level of methodological rigor. I have worked with Gallups editor in chief, Frank Newport, who has a doctorate in sociology from the University of Michigan, and I can testify that he knows more about quantitative methods than I could ever know.
And yet, despite my high regard for the people at Gallup, I have to shake my head in disbelief at how Gallups generic ballot has jumped around recently. And it isnt just the most recent wild bounce.
Gallups weekly track of July 19-25 showed Democrats ahead by 4 points on the generic ballot, but the next week, July 26-Aug. 1, Gallup had the GOP up by 5 points. Thats a huge swing that isnt easy to explain.
In late August, of course, Gallups weekly tracking data showed Republicans opening up a 10-point lead in the generic Congressional ballot, causing more than a small overreaction from veteran political observers. But a week later, Gallups weekly tracking data had the Congressional generic ballot even at 46 percent.
Then, if you compare Gallups generic ballot of June 28-July 3, when Republicans held a 2-point advantage, to its July 12-18 weekly generic ballot, when Democrats held a 6-point edge, you get another uncomfortably large swing without some dramatic news event.
One outlier over two months is understandable. Two is a stretch. Three is something to worry about.
Its worth noting that the NBC News/Wall Street Journal poll, conducted jointly by a Democratic and a Republican pollster, doesnt have dramatic swings in its version of the generic ballot.