You may have seen the headline yesterday: the Gallup poll reported a shift in political identification, such that self-described Democrats now outnumber Republicans by a single point. Editor & Publisher described this as "a (perhaps) historic shift." But wait! Haven't the parties been more or less in a dead heat for some time now? Yes. And what does that say about all the other polls you've seen in the last year, where Democrats typically outnumber Republicans by anywhere from five to thirteen points in the survey sample?

Captain Ed makes the point eloquently:

A one-point difference, especially in a poll of 1,000 adults, falls within the margin of error in any case. It's interesting and revealing to see E&P cast this as a "historic shift" when Gallup reports that this has been unchanged for almost a year. Either E&P has a strong case of analytical illiteracy or they want to put a spin worthy of John McEnroe on what amounts to a lob.

But one issue does arise from this fresh polling, one that undermines practically every political survey published in the past couple of years. If party identification is so close as to be a dead heat, why do polling services routinely underrepresent Republicans? CBS polls routinely overpoll Democrats so badly that their results are hardly worth the effort of analyzing. Their last major poll had a disparity between Democrats and Republicans of thirteen points -- which they corrected to a nine-point difference. Surprise! It found that Bush's approval numbers had dropped!

Here's another that CQ noted earlier this year. Despite undersampling Republicans by 5 points, the NSA terrorist surveillance program managed to garner majority support, with or without warrants. CBS actually had their raw numbers correct in this poll, but "corrected" them through weighting to give Democrats a five-point edge. Again, it should surprise no one that this allowed CBS to report that Bush had suffered another drop in support in May of last year.

Now we have a benchmark against which to measure these polls. Any poll purporting to take the political temperature of the American electorate that doesn't reflect these rather steady numbers should receive the scorn it deserves. The polls that we have seen thus far in 2006 all fail the sampling test...

I actually did see one poll last year that over-sampled Republicans, but that's more or less like catching a glimpse of a passenger pigeon. Baseball genius Bill James has been on my mind, because we're trying to track him down to invite him to be a guest on our radio show. James once wrote that he thought Wrigley Field was partly responsible for the Chicago Cubs' generations-long record of futility, in this way: because Wrigley is so batter-friendly, the Cubs' management has persistently over-rated their own hitting talent, and thus have been too willing to stand pat with rosters that are in fact inadequate. I wonder whether the polls similarly lead the Democrats to overestimate their electoral prospects, thereby contributing to their Cubs-like futility in recent years.

Posted by John at 07:49 AM  |    |