Polls with Large Voter Samples All Favor Romney; Smaller, Less Reliable Polls All Favor Obama. Why?

RealClearPolitics publishes a continuously updated average of all major national presidential polls — as of this evening, their chart looks like this:

(Note that the RCP chart is updated frequently, so that by the time you read this, it may be slightly different.)

But whenever I check the RCP average, including today, I notice something odd: The larger the polling sample size, the more the poll favors Mitt Romney.

I’ve copied the RCP data and pasted it in here in a format that I can re-order. First, here’s how RCP organizes the chart, which they do chronologically, with the most recent poll at the top:

But what if we simply re-ordered the polls not chronologically, but according to sample size, with the largest at the top? This is what it would look like:

Notice the unmistakable trend?

All polls with 1000 or more respondents favor Romney; all polls with smaller than 1000 respondents favor Obama (or are tied).

Statisticians will tell you that the larger the sample size, the more reliable the poll. This fact is reflected in RCP’s “Margin of Error” (MoE) column, which shows a lower margin of error, and thus a greater level of reliability, for the large-sample pro-Romney polls. Each and every pro-Obama poll has a higher margin of error, and is thus less reliable.

These are the facts as they currently stand, and they’ve been true like this almost every day since soon after the first debate when Romney surged in popularity.

But Why?

The question we must now ask ourselves is why only the weak polls with low response rates favor Obama.

We can only speculate. I’ll posit some theories here, but I welcome speculations in the comments section as well.

• Theory #1: Non-respondents refuse to answer because they don’t want to reveal to a stranger that they’re voting for Romney.

As I noted in an earlier post, only 9% of people respond to poll questions: The other 91% either refuse to participate in the poll, or don’t even pick up the phone in the first place. I argued back then that it’s likely the majority of those non-responders are Romney voters, and this new polling trend confirms that thesis. It’s not clear why some pollsters have higher sample sizes than others: it could be that IBD/TIPP, NBC, WashTimes and CBS merely aimed low, and intended to have small poll samples — but it’s more likely that they tried to get more people to reply but had a significant non-response rate, and so “gave up” after reaching some number under 1000. Perhaps something about their methodologies, or the type of people hired to ask the questions, or their wording turned off or offended a certain type of voter inclined to favor Romney, and those people tended to hang up rather than state their preference for Romney. That would mean that IBD/TIPP, NBC, WashTimes and CBS are unintentionally under-sampling Romney voters, leading to their pro-Obama poll results.

• Theory #2: Certain pollsters are purposely publishing polls with high margins of error, so they can later have wiggle room when their pro-Obama slant proves to be inaccurate.

This theories supposes deviousness or malice on the part of certain pollsters: They want to sway the electorate with pro-Obama poll results, but want to give themselves room for plausible deniability if those polls later prove to be inaccurate. If challenged on their dubious published results, they can say, “Yes, our poll seemed to favor Obama, but you see his advantage was within the margin of error, so it’s plausible that it was just a statistical fluke, now that we see in retrospect that Romney won.” But a flaw in this theory is that two of the four pro-Obama polls were published by conservative-leaning media — IBD and Washington Times — who presumably have no interest in helping Obama.

• Theory 3: Sloppy polls simply give inaccurate results.

The smaller the sample, the more loosey-goosey the poll is, and the more likely it will produce an outlier result. But this fails to explain why all the outliers tilt toward Obama.

• Theory 4: The larger sample sizes also tend to be the more recent polls — all this trend reveals is that Romney’s lead is growing.

This is partly true, but there’s a lot of statistical “noise” in the recent/large/pro-Romney correlation. But it may account for some of the trend.

Otherwise, I’m stumped. I’ve love to hear what your theories are about this poll-sample trend. But perhaps it’s best to just bask in the knowledge that this week’s polls confirm that Romney has solidified his lead, and not worry about why some polls are off the mark.