Battle4Liberty

With the presidential primaries in full swing, we are being inundated with constant political analysis and poll results. Polling is mostly a science, but even when done correctly, the interpretations are often suspect. There are lots of problems with polling, but here are three serious issues to consider when digesting the next one.

1. Did the pollsters ask questions that respondents could readily answer? Most exit polls pass this test because they ask voters how they voted and why. Assuming they tell the truth—or that the lies are evenly distributed among the candidates—polls like this are generally reliable if those sampled are representative of the voters at large. If not, the problems are obvious. For example, a poll of younger Democrats would suggest strong support for Sanders while a poll of older Democrats would favor Clinton. Respectable pollsters usually ensure that this is done correctly, although minor errors can result in inaccurate predictions, especially in close races. This is why results can vary widely in polls deemed to be scientific.

Polls that assess hypothetical head-to-head matchups do not pass this test. Likely voters are asked how they would vote given all of the scenarios and the results are supposed to suggest which candidates have the best chance of winning in November. The problems with this approach are legion. It’s not uncommon for voters with a strong primary preference to claim that they will “sit out” the November election if their candidate isn’t the nominee, or possibly crossover to the other party. So much can and will happen in the next eight months that polling hypothetical preferences today is a waste of time.

2. Are the questions worded properly? A pollster with a political ax to grind can modify a word or phrase, or not ask a particular question to paint a skewed picture of reality. For example, if you ask Americans if they favor gun control, you’ll likely get some yes answers from largely pro-gun individuals because they support some, notion of gun control (perhaps convicted felons or the mentally ill). Ask them if they support gun rights, and some largely pro-gun control individuals will likely answer yes because they favor some rights (perhaps hunting rifles). Organizations with a given political position often choose the wording in such instances as a means of influencing public opinion.

3. Are the responses properly interpreted and reported? This isn’t always easy to do, but many reporters and analysts simply don’t understand the science or are willingly misleading their audiences. A simple example is the reporting of gender and ethnicity “gaps” with certain political candidates. Pundits are constantly telling us that Republicans have a problem attracting women voters. Given the rough 50/50 split between women and men and the general balance of power in Washington, it is just as accurate to say that Democrats have a problem attracting men voters. This is rarely suggested, however, because such a conclusion doesn’t follow a predetermined narrative.

A subtler example can be found in recent polls concerning why Republican primary voters select a particular candidate. We are told that those who are most concerned about winning in November overwhelmingly support Rubio, but the evidence is really weak. Voters are typically asked who they selected (or will select) and why. The fact that Rubio voters are more likely than Trump or Cruz voters to identify prospects for winning the general election as the top reason for their vote doesn’t necessarily mean that Trump or Cruz voters aren’t equally convinced that their candidates can win. Indeed, Trump voters are more likely to suggest preference for an outsider as the top reason, but most probably think that their candidate has the best chance of winning in November as well; it’s just not the overriding reason they selected Trump. It could be that Rubio voters don’t really like him very much as a candidate except that they think he has the best chance of winning in November. We are supposed to believe that supporting Rubio is the best choice if “we really want to beat Hillary,” but the alternative explanation I just proposed paints Rubio as a lukewarm choice with a less-than-energetic base of support, thereby making him a less desirable candidate in November. Frankly, I’m not sure which explanation is most accurate, but I’m willing to put both explanations on the table.

I’m not anti polls, but I’m tired of their prominence in lead stories. They are fraught with problems. Most reporters either understand this reality and choose to overlook it or they they don’t fully understand what they’re reporting. Frankly, I think the issue with most is ignorance, but either way we have a serious problem.